Sep 10 13:28:53.597874 ip-10-0-137-53 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 13:28:53.597885 ip-10-0-137-53 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 13:28:53.597892 ip-10-0-137-53 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 13:28:53.598032 ip-10-0-137-53 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 13:29:03.828484 ip-10-0-137-53 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 13:29:03.828504 ip-10-0-137-53 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8ce39a9f535e411c87f3f31801655c29 --
Sep 10 13:30:40.981148 ip-10-0-137-53 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 13:30:41.381320 ip-10-0-137-53 kubenswrapper[2549]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 13:30:41.381320 ip-10-0-137-53 kubenswrapper[2549]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 13:30:41.381320 ip-10-0-137-53 kubenswrapper[2549]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 13:30:41.381320 ip-10-0-137-53 kubenswrapper[2549]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 13:30:41.381320 ip-10-0-137-53 kubenswrapper[2549]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 13:30:41.383322 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.383241    2549 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 13:30:41.387466 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387452    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:30:41.387466 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387466    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387469    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387473    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387476    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387479    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387482    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387485    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387487    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387490    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387493    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387497    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387501    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387504    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387507    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387510    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387513    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387516    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387519    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387530    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:30:41.387527 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387533    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387536    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387539    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387541    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387545    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387549    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387552    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387555    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387558    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387560    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387563    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387566    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387568    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387571    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387579    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387582    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387585    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387587    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387591    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387593    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:30:41.387987 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387596    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387598    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387600    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387603    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387605    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387608    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387610    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387612    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387615    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387617    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387619    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387622    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387624    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387627    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387630    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387633    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387636    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387640    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387643    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:30:41.388489 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387646    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387649    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387652    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387654    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387657    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387660    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387663    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387665    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387668    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387670    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387674    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387676    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387679    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387681    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387684    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387687    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387689    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387692    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387694    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387697    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:30:41.388947 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387699    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387702    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387704    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387707    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387710    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387712    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387715    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387718    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387720    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387723    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387725    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387728    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387730    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387733    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387735    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387738    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387740    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387743    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387745    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.387747    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:30:41.389418 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388461    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388467    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388471    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388474    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388477    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388480    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388483    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388486    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388488    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388491    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388494    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388496    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388499    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388501    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388504    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388507    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388510    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388512    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388515    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388518    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:30:41.389895 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388521    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388523    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388526    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388528    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388530    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388533    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388535    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388538    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388540    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388543    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388546    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388548    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388551    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388553    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388555    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388558    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388560    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388564    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388569    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:30:41.390383 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388573    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388576    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388579    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388582    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388585    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388588    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388590    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388592    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388595    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388598    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388600    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388603    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388606    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388608    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388611    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388614    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388616    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388619    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388621    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388624    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388626    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:30:41.390852 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388630    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388633    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388637    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388640    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388643    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388646    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388649    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388653    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388656    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388659    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388661    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388667    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388669    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388672    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388674    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388677    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388679    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388682    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388684    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388687    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:30:41.391371 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388690    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388693    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388695    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388698    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388700    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388703    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388705    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388708    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388711    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388713    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388716    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388718    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388720    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388723    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388725    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388728    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388730    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388733    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.388735    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389129    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:30:41.391842 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389134    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389138    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389141    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389145    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389148    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389150    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389153    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389155    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389158    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389160    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389163    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389166    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389169    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389171    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389174    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389176    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389179    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389183    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389186    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389190    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:30:41.392342 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389193    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389196    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389199    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389201    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389204    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389207    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389210    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389213    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389216    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389218    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389221    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389224    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389227    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389229    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389232    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389236    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389239    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389242    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389244    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389247    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:30:41.392806 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389250    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389252    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389254    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389258    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389261    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389264    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389268    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389270    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389273    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389275    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389278    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389281    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389283    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389286    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389289    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389291    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389294    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389296    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389298    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389301    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:30:41.393329 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389303    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389306    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389308    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389311    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389313    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389316    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389318    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389322    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389324    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389328    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389330    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389332    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389335    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389338    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389341    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389343    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389346    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389348    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389350    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:30:41.393793 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389353    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389357    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389360    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389363    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389366    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389368    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389371    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389374    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389376    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389378    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389381    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389383    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389386    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389388    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389390    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389393    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389396    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389398    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.389401    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389460    2549 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 13:30:41.394261 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389496    2549 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389504    2549 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389510    2549 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389515    2549 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389519    2549 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389523    2549 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389527    2549 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389530    2549 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389534    2549 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389537    2549 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389540    2549 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389543    2549 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389546    2549 flags.go:64] FLAG: --cgroup-root=""
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389549    2549 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389552    2549 flags.go:64] FLAG: --client-ca-file=""
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389555    2549 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389558    2549 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389562    2549 flags.go:64] FLAG: --cluster-domain=""
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389565    2549 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389568    2549 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389571    2549 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389575    2549 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389579    2549 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 13:30:41.394767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389582    2549 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389585    2549 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389588    2549 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389591    2549 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389595    2549 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389598    2549 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389601    2549 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389604    2549 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389607    2549 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389612    2549 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389614    2549 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389617    2549 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389620    2549 flags.go:64] FLAG: --enable-server="true"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389623    2549 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389627    2549 flags.go:64] FLAG: --event-burst="100"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389631    2549 flags.go:64] FLAG: --event-qps="50"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389634    2549 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389637    2549 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389640    2549 flags.go:64] FLAG: --eviction-hard=""
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389643    2549 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389646    2549 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389649    2549 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389653    2549 flags.go:64] FLAG: --eviction-soft=""
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389655    2549 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389658    2549 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 13:30:41.395319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389661    2549 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389664    2549 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389667    2549 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389670    2549 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389673    2549 flags.go:64] FLAG: --feature-gates=""
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389677    2549 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389680    2549 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389682    2549 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389685    2549 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389688    2549 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389691    2549 flags.go:64] FLAG: --help="false"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389694    2549 flags.go:64] FLAG: --hostname-override="ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389697    2549 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389699    2549 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389702    2549 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389705    2549 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389710    2549 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389713    2549 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389716    2549 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389718    2549 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389721    2549 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389724    2549 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389727    2549 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389730    2549 flags.go:64] FLAG: --kube-reserved=""
Sep 10 13:30:41.395931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389733    2549 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389736    2549 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389740    2549 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389743    2549 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389746    2549 flags.go:64] FLAG: --lock-file=""
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389748    2549 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389751    2549 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389754    2549 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389759    2549 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389762    2549 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389764    2549 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389767    2549 flags.go:64] FLAG: --logging-format="text"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389770    2549 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389773    2549 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389776    2549 flags.go:64] FLAG: --manifest-url=""
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389779    2549 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389783    2549 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389786    2549 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389792    2549 flags.go:64] FLAG: --max-pods="110"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389795    2549 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389798    2549 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389801    2549 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389803    2549 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389807    2549 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389811    2549 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 13:30:41.396698 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389814    2549 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389820    2549 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389823    2549 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389826    2549 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389829    2549 flags.go:64] FLAG: --pod-cidr=""
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389836    2549 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389840    2549 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389843    2549 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389846    2549 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389851    2549 flags.go:64] FLAG: --port="10250"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389854    2549 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389857    2549 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dbf4cbadbe9a1a44"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389860    2549 flags.go:64] FLAG: --qos-reserved=""
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389876    2549 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389881    2549 flags.go:64] FLAG: --register-node="true"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389885    2549 flags.go:64] FLAG: --register-with-taints=""
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389889    2549 flags.go:64] FLAG: --registry-burst="10"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389892    2549 flags.go:64] FLAG: --registry-qps="5"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389895    2549 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389897    2549 flags.go:64] FLAG: --reserved-memory=""
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389901    2549 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389904    2549 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389907    2549 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389910    2549 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389913    2549 flags.go:64] FLAG: --runonce="false"
Sep 10 13:30:41.397721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389916    2549 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389919    2549 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389921    2549 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389924    2549 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389927    2549 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389930    2549 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389933    2549 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389938    2549 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389940    2549 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389943    2549 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389946    2549 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389949    2549 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389952    2549 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389956    2549 flags.go:64] FLAG: --system-cgroups=""
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389958    2549 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389964    2549 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389967    2549 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389970    2549 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389974    2549 flags.go:64] FLAG: --tls-min-version=""
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389977    2549 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389980    2549 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389983    2549 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389985    2549 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389988    2549 flags.go:64] FLAG: --v="2"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389992    2549 flags.go:64] FLAG: --version="false"
Sep 10 13:30:41.398683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389996    2549 flags.go:64] FLAG: --vmodule=""
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.389999    2549 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.390002    2549 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390107    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390111    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390114    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390117    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390120    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390122    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390125    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390128    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390130    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390133    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390137    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390141    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390144    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390147    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390150    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390152    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390154    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:30:41.399618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390157    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390160    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390162    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390166    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390169    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390171    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390174    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390177    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390179    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390182    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390184    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390187    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390189    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390192    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390194    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390197    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390199    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390202    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390204    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:30:41.400131 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390207    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390211    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390215    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390218    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390220    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390223    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390226    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390230    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390234    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390237    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390240    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390242    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390245    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390248    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390251    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390254    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390258    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390261    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390264    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:30:41.400585 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390266    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390269    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390271    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390274    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390276    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390279    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390282    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390290    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390293    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390296    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390298    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390301    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390303    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390306    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390309    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390311    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390314    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390316    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390318    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390321    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:30:41.401052 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390325    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390328    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390330    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390333    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390335    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390338    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390340    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390343    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390345    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390350    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390353    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390355    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390357    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390360    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390362    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390365    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390367    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390370    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390372    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:30:41.401539 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390375    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390377    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390380    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390382    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.390385    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.390934    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.399939    2549 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.399954    2549 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.400955    2549 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.400962    2549 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401016    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401021    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401024    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401028    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401031    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:30:41.402022 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401034    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401037    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401040    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401043    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401047    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401049    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401052    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401055    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401058    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401061    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401064    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401067    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401069    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401072    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401076    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401080    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401083    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401086    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401088    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:30:41.402399 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401091    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401093    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401096    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401099    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401102    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401104    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401106    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401109    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401111    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401114    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401117    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401119    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401122    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401125    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401127    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401130    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401132    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401135    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401138    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401141    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:30:41.402940 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401143    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401146    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401149    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401152    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401154    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401156    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401159    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401161    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401164    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401166    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401169    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401171    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401174    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401176    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401179    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401181    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401184    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401186    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401189    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:30:41.403420 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401192    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401196    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401198    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401201    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401203    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401206    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401209    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401211    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401214    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401216    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401219    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401221    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401223    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401226    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401229    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401232    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401235    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401239    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401241    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401244    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:30:41.403953 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401247    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401250    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401252    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401255    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401258    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401261    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401263    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401266    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401269    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401271    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401274    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401276    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401279    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401282    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401284    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:30:41.404431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401286    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.401291    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401399    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401404    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401409    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401412    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401415    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401417    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401420    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401422    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401425    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401428    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401431    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401433    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:30:41.404779 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401436    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401439    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401441    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401444    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401446    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401449    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401451    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401454    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401457    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401459    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401462    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401464    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401467    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401469    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401472    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401474    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401477    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401480    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401483    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:30:41.405173 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401485    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401488    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401490    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401493    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401495    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401498    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401500    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401502    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401505    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401508    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401510    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401513    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401515    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401517    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401521    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401523    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401526    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401529    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401531    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401534    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:30:41.405618 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401536    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401539    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401541    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401544    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401546    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401549    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401552    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401554    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401557    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401560    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401562    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401564    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401567    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401569    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401572    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401574    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401577    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401579    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401582    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401584    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:30:41.406127 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401586    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401589    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401592    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401594    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401597    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401599    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401601    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401604    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401607    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401610    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401612    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401615    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401617    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401620    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401622    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401625    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401627    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401629    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401633    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401637    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:30:41.406606 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401639    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401642    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401644    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401647    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401649    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401654    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401657    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:41.401659    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.401664    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.401768    2549 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.404676    2549 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.405453    2549 server.go:1019] "Starting client certificate rotation"
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.405555    2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:30:41.407094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.405584    2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:30:41.408039 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.408026    2549 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 13:30:41.409121 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.409102    2549 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 13:30:41.419236 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.419217    2549 log.go:25] "Validated CRI v1 runtime API"
Sep 10 13:30:41.422248 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.422234    2549 log.go:25] "Validated CRI v1 image API"
Sep 10 13:30:41.423169 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.423154    2549 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 13:30:41.427976 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.427947    2549 fs.go:135] Filesystem UUIDs: map[509fe05e-7ef1-4e33-8191-72dacc6073c6:/dev/nvme0n1p3 6d89a2a9-98a9-4785-93cd-86bb6962471c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 10 13:30:41.428048 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.427974    2549 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 13:30:41.432917 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.432792    2549 manager.go:216] Machine: {Timestamp:2026-09-10 13:30:41.431488368 +0000 UTC m=+0.338046390 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200226 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2585a04e1d18cc107eb3e51dd95138 SystemUUID:ec2585a0-4e1d-18cc-107e-b3e51dd95138 BootID:8ce39a9f-535e-411c-87f3-f31801655c29 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:30:bc:3e:71:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:30:bc:3e:71:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:99:38:0c:6d:ae Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 13:30:41.432917 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.432907    2549 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 13:30:41.433068 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.433008    2549 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 13:30:41.435297 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.435276    2549 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 13:30:41.435449 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.435301    2549 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-53.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 13:30:41.436184 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.436173    2549 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 13:30:41.436251 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.436191    2549 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 13:30:41.436251 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.436202    2549 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 13:30:41.436943 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.436930    2549 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 13:30:41.438129 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.438117    2549 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 13:30:41.438888 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.438858    2549 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 13:30:41.440984 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.440970    2549 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 13:30:41.441108 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.441098    2549 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 13:30:41.441161 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.441134    2549 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 13:30:41.441161 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.441154    2549 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 13:30:41.441244 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.441173    2549 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 13:30:41.441244 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.441186    2549 kubelet.go:398] "Adding apiserver pod source"
Sep 10 13:30:41.441244 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.441198    2549 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 13:30:41.443542 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.443155    2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 13:30:41.443542 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.443258    2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 13:30:41.447373 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.447351    2549 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 13:30:41.449563 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.449550    2549 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 13:30:41.449622 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.449579    2549 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 13:30:41.449761 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.449747    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 13:30:41.449802 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.449766    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 13:30:41.449802 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.449772    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 13:30:41.449802 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.449777    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 13:30:41.449802 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.449784    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 13:30:41.449802 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.449790    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 13:30:41.450392 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.450383    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 13:30:41.450392 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.450391    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 13:30:41.450448 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.450398    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 13:30:41.450448 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.450406    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 13:30:41.450448 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.450417    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 13:30:41.450448 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.450429    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 13:30:41.451642 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.451632    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 13:30:41.451683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.451643    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 13:30:41.454088 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.454065    2549 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-53.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:30:41.454714 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.454694    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-53.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 13:30:41.454752 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.454696    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 13:30:41.455458 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.455447    2549 server.go:1268] "Started kubelet"
Sep 10 13:30:41.455558 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.455523    2549 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 13:30:41.455608 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.455537    2549 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 13:30:41.455673 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.455612    2549 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 13:30:41.456365 ip-10-0-137-53 systemd[1]: Started Kubernetes Kubelet.
Sep 10 13:30:41.456778 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.456761    2549 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 13:30:41.457522 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.457505    2549 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 13:30:41.461522 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.461504    2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 13:30:41.461937 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.461919    2549 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 13:30:41.462361 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.462224    2549 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 13:30:41.462361 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.462271    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 10 13:30:41.462603 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.462584    2549 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 13:30:41.462603 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.462603    2549 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 13:30:41.462716 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.462588    2549 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 13:30:41.462716 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.462670    2549 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 13:30:41.462716 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.462676    2549 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 13:30:41.462805 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.462796    2549 factory.go:55] Registering systemd factory
Sep 10 13:30:41.462853 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.462808    2549 factory.go:223] Registration of the systemd container factory successfully
Sep 10 13:30:41.463027 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.463014    2549 factory.go:153] Registering CRI-O factory
Sep 10 13:30:41.463086 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.463030    2549 factory.go:223] Registration of the crio container factory successfully
Sep 10 13:30:41.463086 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.463076    2549 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 13:30:41.463179 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.463092    2549 factory.go:103] Registering Raw factory
Sep 10 13:30:41.463179 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.463106    2549 manager.go:1196] Started watching for new ooms in manager
Sep 10 13:30:41.463511 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.463497    2549 manager.go:349] Starting recovery of all containers
Sep 10 13:30:41.468907 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.468879    2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gzv4l"
Sep 10 13:30:41.472586 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.472542    2549 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-53.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 13:30:41.472586 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.472559    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 13:30:41.473724 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.472513    2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-53.ec2.internal.18d3f8ce3069f96f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-53.ec2.internal,UID:ip-10-0-137-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-53.ec2.internal,},FirstTimestamp:2026-09-10 13:30:41.455421807 +0000 UTC m=+0.361979828,LastTimestamp:2026-09-10 13:30:41.455421807 +0000 UTC m=+0.361979828,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-53.ec2.internal,}"
Sep 10 13:30:41.474982 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.474967    2549 manager.go:354] Recovery completed
Sep 10 13:30:41.478850 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.478832    2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gzv4l"
Sep 10 13:30:41.479165 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.479153    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:30:41.480781 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.480763    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:30:41.480833 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.480794    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:30:41.480833 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.480806    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:30:41.481450 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.481435    2549 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 13:30:41.481450 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.481449    2549 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 13:30:41.481581 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.481463    2549 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 13:30:41.481581 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.481441    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:30:41.482188 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.482173    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:30:41.482236 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.482197    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:30:41.482236 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.482207    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:30:41.482488 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.482475    2549 policy_none.go:49] "None policy: Start"
Sep 10 13:30:41.482532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.482496    2549 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 13:30:41.482532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.482506    2549 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 13:30:41.483613 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.483519    2549 policy_none.go:47] "Start"
Sep 10 13:30:41.509599 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.509584    2549 manager.go:337] "Starting Device Plugin manager"
Sep 10 13:30:41.509689 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.509617    2549 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 13:30:41.509689 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.509627    2549 server.go:87] "Starting device plugin registration server"
Sep 10 13:30:41.509959 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.509944    2549 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 13:30:41.510065 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.509959    2549 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 13:30:41.510126 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.510113    2549 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 13:30:41.510201 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.510191    2549 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 13:30:41.510241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.510202    2549 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 13:30:41.511381 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.511366    2549 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 13:30:41.511470 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.511440    2549 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 10 13:30:41.633600 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.610609    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:30:41.633600 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.611501    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:30:41.633600 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.611527    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:30:41.633600 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.611537    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:30:41.633600 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.611563    2549 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.633600 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.623262    2549 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.633600 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.623281    2549 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-53.ec2.internal\": node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 10 13:30:41.646565 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.646545    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 10 13:30:41.666626 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.666604    2549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 13:30:41.667677 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.667661    2549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 13:30:41.667779 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.667706    2549 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 13:30:41.667779 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.667727    2549 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 13:30:41.667898 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.667799    2549 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 13:30:41.670118 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.670100    2549 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:30:41.747578 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.747556    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 10 13:30:41.768031 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.768011    2549 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"]
Sep 10 13:30:41.768113 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.768075    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:30:41.769717 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.769694    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:30:41.769817 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.769728    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:30:41.769817 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.769740    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:30:41.771127 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.771111    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:30:41.771220 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.771189    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.771220 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.771216    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:30:41.771849 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.771835    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:30:41.771936 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.771859    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:30:41.771936 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.771884    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:30:41.773878 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.773849    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:30:41.773934 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.773897    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:30:41.773934 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.773913    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:30:41.775045 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.775031    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.775088 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.775066    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:30:41.775706 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.775692    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:30:41.775773 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.775719    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:30:41.775773 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.775731    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:30:41.800454 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.800429    2549 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-53.ec2.internal\" not found" node="ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.804545 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.804527    2549 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-53.ec2.internal\" not found" node="ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.848564 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.848545    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 10 13:30:41.865582 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.865562    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb832ba95c432514fb054233d143126e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"cb832ba95c432514fb054233d143126e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.865649 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.865587    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb832ba95c432514fb054233d143126e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"cb832ba95c432514fb054233d143126e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.865649 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.865607    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/651cc6668ff972ae51b87e7dde9a4f79-config\") pod \"kube-apiserver-proxy-ip-10-0-137-53.ec2.internal\" (UID: \"651cc6668ff972ae51b87e7dde9a4f79\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.949331 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:41.949274    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 10 13:30:41.966703 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.966671    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/651cc6668ff972ae51b87e7dde9a4f79-config\") pod \"kube-apiserver-proxy-ip-10-0-137-53.ec2.internal\" (UID: \"651cc6668ff972ae51b87e7dde9a4f79\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.966780 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.966708    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb832ba95c432514fb054233d143126e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"cb832ba95c432514fb054233d143126e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.966780 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.966726    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb832ba95c432514fb054233d143126e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"cb832ba95c432514fb054233d143126e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.966780 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.966761    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb832ba95c432514fb054233d143126e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"cb832ba95c432514fb054233d143126e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.966910 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.966773    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/651cc6668ff972ae51b87e7dde9a4f79-config\") pod \"kube-apiserver-proxy-ip-10-0-137-53.ec2.internal\" (UID: \"651cc6668ff972ae51b87e7dde9a4f79\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:41.966910 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:41.966792    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb832ba95c432514fb054233d143126e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"cb832ba95c432514fb054233d143126e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:42.049830 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.049812    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 10 13:30:42.103297 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.103260    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:42.107666 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.107586    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:42.150283 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.150258    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 10 13:30:42.251227 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.251199    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 10 13:30:42.340635 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.340608    2549 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:30:42.363311 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.363291    2549 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:42.376284 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.376259    2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 13:30:42.378393 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.378380    2549 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 10 13:30:42.388198 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.388184    2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 13:30:42.405141 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.405113    2549 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 13:30:42.405258 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.405239    2549 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 13:30:42.405258 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.405239    2549 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 13:30:42.442252 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.442228    2549 apiserver.go:52] "Watching apiserver"
Sep 10 13:30:42.462715 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.462684    2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 13:30:42.464211 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.464191    2549 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 13:30:42.464597 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.464574    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-vrlf8","openshift-ovn-kubernetes/ovnkube-node-b7sv8","kube-system/global-pull-secret-syncer-7jhj2","kube-system/konnectivity-agent-tdx8g","kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc","openshift-dns/node-resolver-pw9wz","openshift-multus/multus-additional-cni-plugins-dthlr","openshift-multus/multus-wqzv5","openshift-multus/network-metrics-daemon-8qf55","openshift-cluster-node-tuning-operator/tuned-nzhpv","openshift-image-registry/node-ca-7vkmn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal","openshift-network-diagnostics/network-check-target-tqlkj"]
Sep 10 13:30:42.466239 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.466219    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.467601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.467578    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.468743 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.468724    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:42.468838 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.468789    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:30:42.468918 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.468899    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mbdcr\""
Sep 10 13:30:42.469025 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.468887    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 13:30:42.469138 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.469124    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 13:30:42.469172 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.469125    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 13:30:42.470097 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.470079    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:30:42.470266 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.470250    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 13:30:42.470443 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.470428    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 13:30:42.470481 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.470448    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 13:30:42.470712 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.470678    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 13:30:42.470780 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.470758    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 13:30:42.471413 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.471387    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.471512 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.471421    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 13:30:42.471512 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.471421    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xkgg9\""
Sep 10 13:30:42.472663 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.472646    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.473869 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.473852    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.475486 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.475465    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.475693 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.475667    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 13:30:42.477054 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.477029    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 13:30:42.477164 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.477067    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jjwnn\""
Sep 10 13:30:42.477226 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.477028    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 13:30:42.477226 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.477194    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:42.477401 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.477373    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:30:42.478314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.477720    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 13:30:42.478314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.477981    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 13:30:42.478314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.478053    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 13:30:42.478314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.478293    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 13:30:42.478534 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.478338    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rm9b4\""
Sep 10 13:30:42.478534 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.478406    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 13:30:42.478688 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.478671    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 13:30:42.479914 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.478681    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zr66c\""
Sep 10 13:30:42.479914 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.478746    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 13:30:42.479914 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.478765    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nv7wk\""
Sep 10 13:30:42.479914 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.478922    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 13:30:42.479914 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.479261    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 13:30:42.479914 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.479491    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 13:30:42.479914 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.479586    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dzw42\""
Sep 10 13:30:42.479914 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.479758    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.479914 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.479822    2549 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 13:30:42.480593 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.480528    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 13:25:41 +0000 UTC" deadline="2028-06-18 16:54:24.084502974 +0000 UTC"
Sep 10 13:30:42.480868 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.480617    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15531h23m41.603910608s"
Sep 10 13:30:42.481554 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.481535    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.482485 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.482465    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2vwk7\""
Sep 10 13:30:42.482573 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.482552    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 13:30:42.483050 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.483023    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:42.483144 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.483102    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:30:42.483209 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.483166    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 13:30:42.484342 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.484321    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6rzqq\""
Sep 10 13:30:42.484451 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.484359    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 13:30:42.484451 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.484373    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 13:30:42.484451 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.484415    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 13:30:42.508695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.508678    2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qc4qp"
Sep 10 13:30:42.525884 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.525867    2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qc4qp"
Sep 10 13:30:42.530527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.530509    2549 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:30:42.563863 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.563844    2549 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 13:30:42.569438 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569417    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-etc-openvswitch\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.569527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569446    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36a92305-52ee-42ba-b9f4-12f6633781ef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.569527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569470    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-etc-kubernetes\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.569527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569494    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-var-lib-kubelet\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.569527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569518    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-run-ovn\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.569692 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569533    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-env-overrides\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.569692 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569550    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8gnc\" (UniqueName: \"kubernetes.io/projected/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-kube-api-access-h8gnc\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.569692 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569565    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-run-k8s-cni-cncf-io\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.569692 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569589    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-var-lib-kubelet\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.569692 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569638    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:42.569692 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569675    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-var-lib-openvswitch\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.569886 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569709    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36a92305-52ee-42ba-b9f4-12f6633781ef-system-cni-dir\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.569886 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569731    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36a92305-52ee-42ba-b9f4-12f6633781ef-cnibin\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.569886 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569751    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkwvb\" (UniqueName: \"kubernetes.io/projected/2e530ccd-d80f-4199-8b0d-5de767763253-kube-api-access-gkwvb\") pod \"node-ca-7vkmn\" (UID: \"2e530ccd-d80f-4199-8b0d-5de767763253\") " pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.569886 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569769    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36a92305-52ee-42ba-b9f4-12f6633781ef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.569886 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569801    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-run-multus-certs\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.569886 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569816    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e530ccd-d80f-4199-8b0d-5de767763253-host\") pod \"node-ca-7vkmn\" (UID: \"2e530ccd-d80f-4199-8b0d-5de767763253\") " pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.569886 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569851    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569903    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-sys-fs\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569922    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v67bb\" (UniqueName: \"kubernetes.io/projected/c3ef299f-589b-4956-ab88-113c8cb03f26-kube-api-access-v67bb\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569940    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-kubelet\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569960    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.569978    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-var-lib-cni-bin\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570008    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-sysctl-conf\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570042    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-lib-modules\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570072    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/558a818a-f2a0-4a6c-aa06-ec3683eab67b-host-slash\") pod \"iptables-alerter-vrlf8\" (UID: \"558a818a-f2a0-4a6c-aa06-ec3683eab67b\") " pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570097    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25mvc\" (UniqueName: \"kubernetes.io/projected/ffad911d-9cef-4136-aeb5-ed9c6352add7-kube-api-access-25mvc\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570119    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-run-netns\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.570148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570145    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-ovn-node-metrics-cert\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570165    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-run-netns\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570190    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-daemon-config\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570209    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfj6p\" (UniqueName: \"kubernetes.io/projected/3986726d-5c3a-4a89-a469-58d7d779f0a5-kube-api-access-zfj6p\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570223    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-sysctl-d\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570255    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-host\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570282    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e530ccd-d80f-4199-8b0d-5de767763253-serviceca\") pod \"node-ca-7vkmn\" (UID: \"2e530ccd-d80f-4199-8b0d-5de767763253\") " pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570304    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-systemd-units\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570325    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-cnibin\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570343    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-systemd\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570369    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-log-socket\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570393    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-sysconfig\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570411    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-run\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570426    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9brmp\" (UniqueName: \"kubernetes.io/projected/558a818a-f2a0-4a6c-aa06-ec3683eab67b-kube-api-access-9brmp\") pod \"iptables-alerter-vrlf8\" (UID: \"558a818a-f2a0-4a6c-aa06-ec3683eab67b\") " pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570450    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnl56\" (UniqueName: \"kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56\") pod \"network-check-target-tqlkj\" (UID: \"88d95652-29a9-4d30-b0e0-4795dc5e5077\") " pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570473    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-system-cni-dir\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.570532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570523    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-socket-dir-parent\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570548    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46769783-6500-4cc8-8447-587bed957524-tmp\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570571    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-etc-selinux\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570598    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-slash\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570613    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-node-log\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570633    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dzxq\" (UniqueName: \"kubernetes.io/projected/dc277d0a-0d17-499c-96e8-50582c9a2655-kube-api-access-4dzxq\") pod \"node-resolver-pw9wz\" (UID: \"dc277d0a-0d17-499c-96e8-50582c9a2655\") " pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570665    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-hostroot\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570696    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/46769783-6500-4cc8-8447-587bed957524-etc-tuned\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570730    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-device-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570751    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-run-systemd\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570795    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-run-ovn-kubernetes\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570831    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-cni-bin\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570858    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36a92305-52ee-42ba-b9f4-12f6633781ef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570904    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-var-lib-cni-multus\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570934    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-modprobe-d\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570971    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jksj\" (UniqueName: \"kubernetes.io/projected/46769783-6500-4cc8-8447-587bed957524-kube-api-access-7jksj\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.571099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.570991    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc277d0a-0d17-499c-96e8-50582c9a2655-tmp-dir\") pod \"node-resolver-pw9wz\" (UID: \"dc277d0a-0d17-499c-96e8-50582c9a2655\") " pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571007    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36a92305-52ee-42ba-b9f4-12f6633781ef-cni-binary-copy\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571037    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-kubelet-config\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571053    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-socket-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571067    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc277d0a-0d17-499c-96e8-50582c9a2655-hosts-file\") pod \"node-resolver-pw9wz\" (UID: \"dc277d0a-0d17-499c-96e8-50582c9a2655\") " pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571083    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-cni-dir\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571112    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7glmt\" (UniqueName: \"kubernetes.io/projected/36a92305-52ee-42ba-b9f4-12f6633781ef-kube-api-access-7glmt\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571134    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-sys\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571150    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-ovnkube-config\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571165    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-conf-dir\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571185    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/69d18837-83dd-4960-a093-eabc75e22ddb-agent-certs\") pod \"konnectivity-agent-tdx8g\" (UID: \"69d18837-83dd-4960-a093-eabc75e22ddb\") " pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571208    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571228    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-run-openvswitch\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571244    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-ovnkube-script-lib\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571262    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3986726d-5c3a-4a89-a469-58d7d779f0a5-cni-binary-copy\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571290    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-kubernetes\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.571601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571319    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/558a818a-f2a0-4a6c-aa06-ec3683eab67b-iptables-alerter-script\") pod \"iptables-alerter-vrlf8\" (UID: \"558a818a-f2a0-4a6c-aa06-ec3683eab67b\") " pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.572148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571334    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-dbus\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:42.572148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571351    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-registration-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.572148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571369    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-cni-netd\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.572148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.571386    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/69d18837-83dd-4960-a093-eabc75e22ddb-konnectivity-ca\") pod \"konnectivity-agent-tdx8g\" (UID: \"69d18837-83dd-4960-a093-eabc75e22ddb\") " pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:30:42.671590 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671556    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-var-lib-cni-multus\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.671590 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671586    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-modprobe-d\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671605    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jksj\" (UniqueName: \"kubernetes.io/projected/46769783-6500-4cc8-8447-587bed957524-kube-api-access-7jksj\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671622    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc277d0a-0d17-499c-96e8-50582c9a2655-tmp-dir\") pod \"node-resolver-pw9wz\" (UID: \"dc277d0a-0d17-499c-96e8-50582c9a2655\") " pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671646    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36a92305-52ee-42ba-b9f4-12f6633781ef-cni-binary-copy\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671670    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-kubelet-config\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671641    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-var-lib-cni-multus\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671711    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-socket-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671734    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc277d0a-0d17-499c-96e8-50582c9a2655-hosts-file\") pod \"node-resolver-pw9wz\" (UID: \"dc277d0a-0d17-499c-96e8-50582c9a2655\") " pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671758    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-cni-dir\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671783    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7glmt\" (UniqueName: \"kubernetes.io/projected/36a92305-52ee-42ba-b9f4-12f6633781ef-kube-api-access-7glmt\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671790    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-kubelet-config\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671806    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-sys\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671830    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-ovnkube-config\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671854    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-conf-dir\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.671862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671860    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc277d0a-0d17-499c-96e8-50582c9a2655-hosts-file\") pod \"node-resolver-pw9wz\" (UID: \"dc277d0a-0d17-499c-96e8-50582c9a2655\") " pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671878    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/69d18837-83dd-4960-a093-eabc75e22ddb-agent-certs\") pod \"konnectivity-agent-tdx8g\" (UID: \"69d18837-83dd-4960-a093-eabc75e22ddb\") " pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671862    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-socket-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671925    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc277d0a-0d17-499c-96e8-50582c9a2655-tmp-dir\") pod \"node-resolver-pw9wz\" (UID: \"dc277d0a-0d17-499c-96e8-50582c9a2655\") " pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671931    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-cni-dir\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671947    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-conf-dir\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.671978    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672004    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-run-openvswitch\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672003    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-sys\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672035    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-ovnkube-script-lib\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672061    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3986726d-5c3a-4a89-a469-58d7d779f0a5-cni-binary-copy\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672070    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672093    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-kubernetes\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672112    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/558a818a-f2a0-4a6c-aa06-ec3683eab67b-iptables-alerter-script\") pod \"iptables-alerter-vrlf8\" (UID: \"558a818a-f2a0-4a6c-aa06-ec3683eab67b\") " pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672126    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-dbus\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672142    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-registration-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672157    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-cni-netd\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.672560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672173    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/69d18837-83dd-4960-a093-eabc75e22ddb-konnectivity-ca\") pod \"konnectivity-agent-tdx8g\" (UID: \"69d18837-83dd-4960-a093-eabc75e22ddb\") " pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672190    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-etc-openvswitch\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672207    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36a92305-52ee-42ba-b9f4-12f6633781ef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672221    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-dbus\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672237    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-etc-kubernetes\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672271    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-var-lib-kubelet\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672294    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-run-ovn\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672316    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-env-overrides\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672328    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-kubernetes\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672357    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36a92305-52ee-42ba-b9f4-12f6633781ef-cni-binary-copy\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672421    2549 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672492    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-ovnkube-config\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672523    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-modprobe-d\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672563    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-run-openvswitch\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672581    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-etc-openvswitch\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672624    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-cni-netd\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672640    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-registration-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672702    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-var-lib-kubelet\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.673305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672726    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-env-overrides\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672748    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-run-ovn\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672762    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3986726d-5c3a-4a89-a469-58d7d779f0a5-cni-binary-copy\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672771    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8gnc\" (UniqueName: \"kubernetes.io/projected/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-kube-api-access-h8gnc\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672785    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-etc-kubernetes\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672808    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-run-k8s-cni-cncf-io\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672838    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-var-lib-kubelet\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672866    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672892    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-var-lib-openvswitch\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672914    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-run-k8s-cni-cncf-io\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672919    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36a92305-52ee-42ba-b9f4-12f6633781ef-system-cni-dir\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672960    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36a92305-52ee-42ba-b9f4-12f6633781ef-cnibin\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672974    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36a92305-52ee-42ba-b9f4-12f6633781ef-system-cni-dir\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.672988    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkwvb\" (UniqueName: \"kubernetes.io/projected/2e530ccd-d80f-4199-8b0d-5de767763253-kube-api-access-gkwvb\") pod \"node-ca-7vkmn\" (UID: \"2e530ccd-d80f-4199-8b0d-5de767763253\") " pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673014    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36a92305-52ee-42ba-b9f4-12f6633781ef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673042    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-run-multus-certs\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.673048    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:42.674076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673067    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e530ccd-d80f-4199-8b0d-5de767763253-host\") pod \"node-ca-7vkmn\" (UID: \"2e530ccd-d80f-4199-8b0d-5de767763253\") " pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.673099    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs podName:ffad911d-9cef-4136-aeb5-ed9c6352add7 nodeName:}" failed. No retries permitted until 2026-09-10 13:30:43.173084119 +0000 UTC m=+2.079642129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs") pod "network-metrics-daemon-8qf55" (UID: "ffad911d-9cef-4136-aeb5-ed9c6352add7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673102    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/69d18837-83dd-4960-a093-eabc75e22ddb-konnectivity-ca\") pod \"konnectivity-agent-tdx8g\" (UID: \"69d18837-83dd-4960-a093-eabc75e22ddb\") " pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673111    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e530ccd-d80f-4199-8b0d-5de767763253-host\") pod \"node-ca-7vkmn\" (UID: \"2e530ccd-d80f-4199-8b0d-5de767763253\") " pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673117    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673017    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-var-lib-openvswitch\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673049    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36a92305-52ee-42ba-b9f4-12f6633781ef-cnibin\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673147    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36a92305-52ee-42ba-b9f4-12f6633781ef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673197    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-var-lib-kubelet\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673241    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-run-multus-certs\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673244    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36a92305-52ee-42ba-b9f4-12f6633781ef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673244    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-sys-fs\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673153    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-sys-fs\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.673347    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673367    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v67bb\" (UniqueName: \"kubernetes.io/projected/c3ef299f-589b-4956-ab88-113c8cb03f26-kube-api-access-v67bb\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673400    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-kubelet\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.674922 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.673411    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret podName:fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e nodeName:}" failed. No retries permitted until 2026-09-10 13:30:43.173397686 +0000 UTC m=+2.079955701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret") pod "global-pull-secret-syncer-7jhj2" (UID: "fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673446    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-kubelet\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673446    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673482    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-var-lib-cni-bin\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673487    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673521    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-sysctl-conf\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673527    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-var-lib-cni-bin\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673548    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-lib-modules\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673572    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/558a818a-f2a0-4a6c-aa06-ec3683eab67b-host-slash\") pod \"iptables-alerter-vrlf8\" (UID: \"558a818a-f2a0-4a6c-aa06-ec3683eab67b\") " pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673579    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-ovnkube-script-lib\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673597    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25mvc\" (UniqueName: \"kubernetes.io/projected/ffad911d-9cef-4136-aeb5-ed9c6352add7-kube-api-access-25mvc\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673650    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/558a818a-f2a0-4a6c-aa06-ec3683eab67b-iptables-alerter-script\") pod \"iptables-alerter-vrlf8\" (UID: \"558a818a-f2a0-4a6c-aa06-ec3683eab67b\") " pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673661    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/558a818a-f2a0-4a6c-aa06-ec3683eab67b-host-slash\") pod \"iptables-alerter-vrlf8\" (UID: \"558a818a-f2a0-4a6c-aa06-ec3683eab67b\") " pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673654    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-sysctl-conf\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673688    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-run-netns\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673695    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-lib-modules\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673704    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-run-netns\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.675741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673721    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-ovn-node-metrics-cert\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673758    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-run-netns\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673782    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-daemon-config\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673806    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfj6p\" (UniqueName: \"kubernetes.io/projected/3986726d-5c3a-4a89-a469-58d7d779f0a5-kube-api-access-zfj6p\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673814    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-host-run-netns\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673828    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-sysctl-d\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673853    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-host\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673877    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e530ccd-d80f-4199-8b0d-5de767763253-serviceca\") pod \"node-ca-7vkmn\" (UID: \"2e530ccd-d80f-4199-8b0d-5de767763253\") " pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673914    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-systemd-units\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673921    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-host\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673942    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-cnibin\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673941    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-sysctl-d\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.673984    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-systemd\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674002    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-cnibin\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674004    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-systemd-units\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674019    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-log-socket\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674065    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-systemd\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674073    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-sysconfig\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.676470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674096    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-run\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674109    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-log-socket\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674120    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9brmp\" (UniqueName: \"kubernetes.io/projected/558a818a-f2a0-4a6c-aa06-ec3683eab67b-kube-api-access-9brmp\") pod \"iptables-alerter-vrlf8\" (UID: \"558a818a-f2a0-4a6c-aa06-ec3683eab67b\") " pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674149    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnl56\" (UniqueName: \"kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56\") pod \"network-check-target-tqlkj\" (UID: \"88d95652-29a9-4d30-b0e0-4795dc5e5077\") " pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674156    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-etc-sysconfig\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674207    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-system-cni-dir\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674246    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-socket-dir-parent\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674270    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46769783-6500-4cc8-8447-587bed957524-tmp\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674294    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-etc-selinux\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674321    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-slash\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674359    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-node-log\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674383    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dzxq\" (UniqueName: \"kubernetes.io/projected/dc277d0a-0d17-499c-96e8-50582c9a2655-kube-api-access-4dzxq\") pod \"node-resolver-pw9wz\" (UID: \"dc277d0a-0d17-499c-96e8-50582c9a2655\") " pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674416    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-hostroot\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674441    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/46769783-6500-4cc8-8447-587bed957524-etc-tuned\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674458    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-daemon-config\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674466    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e530ccd-d80f-4199-8b0d-5de767763253-serviceca\") pod \"node-ca-7vkmn\" (UID: \"2e530ccd-d80f-4199-8b0d-5de767763253\") " pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674457    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-multus-socket-dir-parent\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674202    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/46769783-6500-4cc8-8447-587bed957524-run\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.677049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674473    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-device-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674530    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-device-dir\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674535    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-run-systemd\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674568    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-run-ovn-kubernetes\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674576    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-system-cni-dir\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674614    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-run-systemd\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674616    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-run-ovn-kubernetes\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674636    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-cni-bin\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674668    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36a92305-52ee-42ba-b9f4-12f6633781ef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674682    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-node-log\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674709    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-cni-bin\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674739    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c3ef299f-589b-4956-ab88-113c8cb03f26-etc-selinux\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674766    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3986726d-5c3a-4a89-a469-58d7d779f0a5-hostroot\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.674785    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-host-slash\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.675133    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36a92305-52ee-42ba-b9f4-12f6633781ef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.676530    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-ovn-node-metrics-cert\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.676698    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/69d18837-83dd-4960-a093-eabc75e22ddb-agent-certs\") pod \"konnectivity-agent-tdx8g\" (UID: \"69d18837-83dd-4960-a093-eabc75e22ddb\") " pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:30:42.677862 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.676876    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46769783-6500-4cc8-8447-587bed957524-tmp\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.678559 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.678249    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/46769783-6500-4cc8-8447-587bed957524-etc-tuned\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.682288 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.682260    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:30:42.682288 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.682285    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:30:42.682456 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.682298    2549 projected.go:196] Error preparing data for projected volume kube-api-access-qnl56 for pod openshift-network-diagnostics/network-check-target-tqlkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:42.682456 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:42.682354    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56 podName:88d95652-29a9-4d30-b0e0-4795dc5e5077 nodeName:}" failed. No retries permitted until 2026-09-10 13:30:43.182335656 +0000 UTC m=+2.088893680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qnl56" (UniqueName: "kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56") pod "network-check-target-tqlkj" (UID: "88d95652-29a9-4d30-b0e0-4795dc5e5077") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:42.682719 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.682689    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7glmt\" (UniqueName: \"kubernetes.io/projected/36a92305-52ee-42ba-b9f4-12f6633781ef-kube-api-access-7glmt\") pod \"multus-additional-cni-plugins-dthlr\" (UID: \"36a92305-52ee-42ba-b9f4-12f6633781ef\") " pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.683093 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.683072    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8gnc\" (UniqueName: \"kubernetes.io/projected/96a4f241-7d7a-45e1-b838-e85ce3b4dad1-kube-api-access-h8gnc\") pod \"ovnkube-node-b7sv8\" (UID: \"96a4f241-7d7a-45e1-b838-e85ce3b4dad1\") " pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.683304 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.683278    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jksj\" (UniqueName: \"kubernetes.io/projected/46769783-6500-4cc8-8447-587bed957524-kube-api-access-7jksj\") pod \"tuned-nzhpv\" (UID: \"46769783-6500-4cc8-8447-587bed957524\") " pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.684859 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.684832    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfj6p\" (UniqueName: \"kubernetes.io/projected/3986726d-5c3a-4a89-a469-58d7d779f0a5-kube-api-access-zfj6p\") pod \"multus-wqzv5\" (UID: \"3986726d-5c3a-4a89-a469-58d7d779f0a5\") " pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.685386 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.685359    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25mvc\" (UniqueName: \"kubernetes.io/projected/ffad911d-9cef-4136-aeb5-ed9c6352add7-kube-api-access-25mvc\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:42.685505 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.685483    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9brmp\" (UniqueName: \"kubernetes.io/projected/558a818a-f2a0-4a6c-aa06-ec3683eab67b-kube-api-access-9brmp\") pod \"iptables-alerter-vrlf8\" (UID: \"558a818a-f2a0-4a6c-aa06-ec3683eab67b\") " pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.685973 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.685935    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkwvb\" (UniqueName: \"kubernetes.io/projected/2e530ccd-d80f-4199-8b0d-5de767763253-kube-api-access-gkwvb\") pod \"node-ca-7vkmn\" (UID: \"2e530ccd-d80f-4199-8b0d-5de767763253\") " pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.686643 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.686618    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v67bb\" (UniqueName: \"kubernetes.io/projected/c3ef299f-589b-4956-ab88-113c8cb03f26-kube-api-access-v67bb\") pod \"aws-ebs-csi-driver-node-m8xtc\" (UID: \"c3ef299f-589b-4956-ab88-113c8cb03f26\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.686716 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.686678    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dzxq\" (UniqueName: \"kubernetes.io/projected/dc277d0a-0d17-499c-96e8-50582c9a2655-kube-api-access-4dzxq\") pod \"node-resolver-pw9wz\" (UID: \"dc277d0a-0d17-499c-96e8-50582c9a2655\") " pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.689677 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.689649    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod651cc6668ff972ae51b87e7dde9a4f79.slice/crio-e8d875e9a7b566ad8ed8b3f07a1cecf23c6b238c8f2e11e91b98e2600c8de932 WatchSource:0}: Error finding container e8d875e9a7b566ad8ed8b3f07a1cecf23c6b238c8f2e11e91b98e2600c8de932: Status 404 returned error can't find the container with id e8d875e9a7b566ad8ed8b3f07a1cecf23c6b238c8f2e11e91b98e2600c8de932
Sep 10 13:30:42.695032 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.695016    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:30:42.797995 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.797920    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vrlf8"
Sep 10 13:30:42.806248 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.806226    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558a818a_f2a0_4a6c_aa06_ec3683eab67b.slice/crio-6e3446432e3a03101fdfc2c664cc2e2c0e5e2be4222a0555bc9e3373add5681c WatchSource:0}: Error finding container 6e3446432e3a03101fdfc2c664cc2e2c0e5e2be4222a0555bc9e3373add5681c: Status 404 returned error can't find the container with id 6e3446432e3a03101fdfc2c664cc2e2c0e5e2be4222a0555bc9e3373add5681c
Sep 10 13:30:42.814620 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.814598    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:30:42.821598 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.821573    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a4f241_7d7a_45e1_b838_e85ce3b4dad1.slice/crio-749ed660d3fe036324d0a19c293dfbbb7366d442dc76935f0336c07dcebac7c5 WatchSource:0}: Error finding container 749ed660d3fe036324d0a19c293dfbbb7366d442dc76935f0336c07dcebac7c5: Status 404 returned error can't find the container with id 749ed660d3fe036324d0a19c293dfbbb7366d442dc76935f0336c07dcebac7c5
Sep 10 13:30:42.837503 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.837483    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:30:42.842267 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.842251    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc"
Sep 10 13:30:42.846235 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.846215    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69d18837_83dd_4960_a093_eabc75e22ddb.slice/crio-3b5bd059dbc83532b61a27fabf066c769b8f4473290b716fd4e432c2b00cd83f WatchSource:0}: Error finding container 3b5bd059dbc83532b61a27fabf066c769b8f4473290b716fd4e432c2b00cd83f: Status 404 returned error can't find the container with id 3b5bd059dbc83532b61a27fabf066c769b8f4473290b716fd4e432c2b00cd83f
Sep 10 13:30:42.846750 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.846731    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pw9wz"
Sep 10 13:30:42.850490 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.850466    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ef299f_589b_4956_ab88_113c8cb03f26.slice/crio-41c76965bec91a3f511ab14f5cb9d690ea98699a223257121021e7e029e10698 WatchSource:0}: Error finding container 41c76965bec91a3f511ab14f5cb9d690ea98699a223257121021e7e029e10698: Status 404 returned error can't find the container with id 41c76965bec91a3f511ab14f5cb9d690ea98699a223257121021e7e029e10698
Sep 10 13:30:42.852523 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.852497    2549 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:30:42.856968 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.856950    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc277d0a_0d17_499c_96e8_50582c9a2655.slice/crio-5a8476353eb997d65ee82261902196623784d8f4e5f335ee844a7b1a969f6553 WatchSource:0}: Error finding container 5a8476353eb997d65ee82261902196623784d8f4e5f335ee844a7b1a969f6553: Status 404 returned error can't find the container with id 5a8476353eb997d65ee82261902196623784d8f4e5f335ee844a7b1a969f6553
Sep 10 13:30:42.875719 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.875696    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dthlr"
Sep 10 13:30:42.882830 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.882802    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a92305_52ee_42ba_b9f4_12f6633781ef.slice/crio-f29b9170cf594ec1a517649660c79cf2ea533f7d91ed59cb609f9cc123c8f1c8 WatchSource:0}: Error finding container f29b9170cf594ec1a517649660c79cf2ea533f7d91ed59cb609f9cc123c8f1c8: Status 404 returned error can't find the container with id f29b9170cf594ec1a517649660c79cf2ea533f7d91ed59cb609f9cc123c8f1c8
Sep 10 13:30:42.883938 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.883918    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wqzv5"
Sep 10 13:30:42.889051 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.889023    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nzhpv"
Sep 10 13:30:42.893878 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.893858    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3986726d_5c3a_4a89_a469_58d7d779f0a5.slice/crio-e2fb239c0a2e1d73aa4cb366b5725ea6377fc2f23ae6f46ede551b1f61e38c5f WatchSource:0}: Error finding container e2fb239c0a2e1d73aa4cb366b5725ea6377fc2f23ae6f46ede551b1f61e38c5f: Status 404 returned error can't find the container with id e2fb239c0a2e1d73aa4cb366b5725ea6377fc2f23ae6f46ede551b1f61e38c5f
Sep 10 13:30:42.896961 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.896886    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46769783_6500_4cc8_8447_587bed957524.slice/crio-ae7643875fac04a8e5c224f98e594349b684a66466450d2ef4930ca152b7c3f6 WatchSource:0}: Error finding container ae7643875fac04a8e5c224f98e594349b684a66466450d2ef4930ca152b7c3f6: Status 404 returned error can't find the container with id ae7643875fac04a8e5c224f98e594349b684a66466450d2ef4930ca152b7c3f6
Sep 10 13:30:42.908052 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:42.908033    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vkmn"
Sep 10 13:30:42.915635 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.915615    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e530ccd_d80f_4199_8b0d_5de767763253.slice/crio-c61dcef3db617f892a77a92f3028e06de2243f72a8c6c1556e2a0cc4532291ca WatchSource:0}: Error finding container c61dcef3db617f892a77a92f3028e06de2243f72a8c6c1556e2a0cc4532291ca: Status 404 returned error can't find the container with id c61dcef3db617f892a77a92f3028e06de2243f72a8c6c1556e2a0cc4532291ca
Sep 10 13:30:42.941526 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:30:42.941503    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb832ba95c432514fb054233d143126e.slice/crio-e8486df6e25f49b0de3202b6ce9fdff8081e690482cabebb1b388d26b0cca1d7 WatchSource:0}: Error finding container e8486df6e25f49b0de3202b6ce9fdff8081e690482cabebb1b388d26b0cca1d7: Status 404 returned error can't find the container with id e8486df6e25f49b0de3202b6ce9fdff8081e690482cabebb1b388d26b0cca1d7
Sep 10 13:30:43.178810 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.178730    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:43.178952 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.178824    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:43.178952 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:43.178893    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:43.178952 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:43.178917    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:43.179062 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:43.178954    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret podName:fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e nodeName:}" failed. No retries permitted until 2026-09-10 13:30:44.178939398 +0000 UTC m=+3.085497412 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret") pod "global-pull-secret-syncer-7jhj2" (UID: "fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:43.179062 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:43.178970    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs podName:ffad911d-9cef-4136-aeb5-ed9c6352add7 nodeName:}" failed. No retries permitted until 2026-09-10 13:30:44.178962245 +0000 UTC m=+3.085520256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs") pod "network-metrics-daemon-8qf55" (UID: "ffad911d-9cef-4136-aeb5-ed9c6352add7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:43.280203 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.280173    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnl56\" (UniqueName: \"kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56\") pod \"network-check-target-tqlkj\" (UID: \"88d95652-29a9-4d30-b0e0-4795dc5e5077\") " pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:43.280365 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:43.280322    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:30:43.280365 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:43.280341    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:30:43.280365 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:43.280350    2549 projected.go:196] Error preparing data for projected volume kube-api-access-qnl56 for pod openshift-network-diagnostics/network-check-target-tqlkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:43.280497 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:43.280405    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56 podName:88d95652-29a9-4d30-b0e0-4795dc5e5077 nodeName:}" failed. No retries permitted until 2026-09-10 13:30:44.280386437 +0000 UTC m=+3.186944465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qnl56" (UniqueName: "kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56") pod "network-check-target-tqlkj" (UID: "88d95652-29a9-4d30-b0e0-4795dc5e5077") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:43.528421 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.528302    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 13:25:42 +0000 UTC" deadline="2028-04-15 04:44:43.360557961 +0000 UTC"
Sep 10 13:30:43.528421 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.528371    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13983h13m59.832192146s"
Sep 10 13:30:43.678959 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.678874    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vkmn" event={"ID":"2e530ccd-d80f-4199-8b0d-5de767763253","Type":"ContainerStarted","Data":"c61dcef3db617f892a77a92f3028e06de2243f72a8c6c1556e2a0cc4532291ca"}
Sep 10 13:30:43.683766 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.683733    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nzhpv" event={"ID":"46769783-6500-4cc8-8447-587bed957524","Type":"ContainerStarted","Data":"ae7643875fac04a8e5c224f98e594349b684a66466450d2ef4930ca152b7c3f6"}
Sep 10 13:30:43.685482 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.685423    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wqzv5" event={"ID":"3986726d-5c3a-4a89-a469-58d7d779f0a5","Type":"ContainerStarted","Data":"e2fb239c0a2e1d73aa4cb366b5725ea6377fc2f23ae6f46ede551b1f61e38c5f"}
Sep 10 13:30:43.688552 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.688526    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dthlr" event={"ID":"36a92305-52ee-42ba-b9f4-12f6633781ef","Type":"ContainerStarted","Data":"f29b9170cf594ec1a517649660c79cf2ea533f7d91ed59cb609f9cc123c8f1c8"}
Sep 10 13:30:43.691839 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.691811    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc" event={"ID":"c3ef299f-589b-4956-ab88-113c8cb03f26","Type":"ContainerStarted","Data":"41c76965bec91a3f511ab14f5cb9d690ea98699a223257121021e7e029e10698"}
Sep 10 13:30:43.694775 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.694750    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tdx8g" event={"ID":"69d18837-83dd-4960-a093-eabc75e22ddb","Type":"ContainerStarted","Data":"3b5bd059dbc83532b61a27fabf066c769b8f4473290b716fd4e432c2b00cd83f"}
Sep 10 13:30:43.696543 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.696519    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vrlf8" event={"ID":"558a818a-f2a0-4a6c-aa06-ec3683eab67b","Type":"ContainerStarted","Data":"6e3446432e3a03101fdfc2c664cc2e2c0e5e2be4222a0555bc9e3373add5681c"}
Sep 10 13:30:43.700930 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.700836    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pw9wz" event={"ID":"dc277d0a-0d17-499c-96e8-50582c9a2655","Type":"ContainerStarted","Data":"5a8476353eb997d65ee82261902196623784d8f4e5f335ee844a7b1a969f6553"}
Sep 10 13:30:43.706643 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.706542    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" event={"ID":"96a4f241-7d7a-45e1-b838-e85ce3b4dad1","Type":"ContainerStarted","Data":"749ed660d3fe036324d0a19c293dfbbb7366d442dc76935f0336c07dcebac7c5"}
Sep 10 13:30:43.716375 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.716331    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal" event={"ID":"651cc6668ff972ae51b87e7dde9a4f79","Type":"ContainerStarted","Data":"e8d875e9a7b566ad8ed8b3f07a1cecf23c6b238c8f2e11e91b98e2600c8de932"}
Sep 10 13:30:43.718546 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.718521    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal" event={"ID":"cb832ba95c432514fb054233d143126e","Type":"ContainerStarted","Data":"e8486df6e25f49b0de3202b6ce9fdff8081e690482cabebb1b388d26b0cca1d7"}
Sep 10 13:30:43.836577 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:43.836496    2549 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:30:44.187361 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:44.187276    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:44.187361 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:44.187335    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:44.187592 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.187501    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:44.187592 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.187569    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret podName:fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e nodeName:}" failed. No retries permitted until 2026-09-10 13:30:46.187551185 +0000 UTC m=+5.094109210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret") pod "global-pull-secret-syncer-7jhj2" (UID: "fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:44.188021 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.187996    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:44.188133 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.188054    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs podName:ffad911d-9cef-4136-aeb5-ed9c6352add7 nodeName:}" failed. No retries permitted until 2026-09-10 13:30:46.188037164 +0000 UTC m=+5.094595180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs") pod "network-metrics-daemon-8qf55" (UID: "ffad911d-9cef-4136-aeb5-ed9c6352add7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:44.288390 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:44.288347    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnl56\" (UniqueName: \"kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56\") pod \"network-check-target-tqlkj\" (UID: \"88d95652-29a9-4d30-b0e0-4795dc5e5077\") " pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:44.288689 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.288526    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:30:44.288689 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.288548    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:30:44.288689 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.288562    2549 projected.go:196] Error preparing data for projected volume kube-api-access-qnl56 for pod openshift-network-diagnostics/network-check-target-tqlkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:44.288689 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.288631    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56 podName:88d95652-29a9-4d30-b0e0-4795dc5e5077 nodeName:}" failed. No retries permitted until 2026-09-10 13:30:46.288611036 +0000 UTC m=+5.195169053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qnl56" (UniqueName: "kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56") pod "network-check-target-tqlkj" (UID: "88d95652-29a9-4d30-b0e0-4795dc5e5077") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:44.528614 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:44.528498    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 13:25:42 +0000 UTC" deadline="2028-04-18 21:21:37.848724781 +0000 UTC"
Sep 10 13:30:44.528614 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:44.528539    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14071h50m53.320190819s"
Sep 10 13:30:44.668891 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:44.668853    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:44.669094 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.669012    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:30:44.669470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:44.669451    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:44.669584 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.669565    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:30:44.669653 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:44.669625    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:44.669707 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:44.669694    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:30:44.813011 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:44.812915    2549 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:30:46.210577 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:46.210313    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:46.211078 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.210492    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:46.211078 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:46.210688    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:46.211078 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.210730    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret podName:fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e nodeName:}" failed. No retries permitted until 2026-09-10 13:30:50.210706091 +0000 UTC m=+9.117264115 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret") pod "global-pull-secret-syncer-7jhj2" (UID: "fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:46.211078 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.210805    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:46.211078 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.210866    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs podName:ffad911d-9cef-4136-aeb5-ed9c6352add7 nodeName:}" failed. No retries permitted until 2026-09-10 13:30:50.210849254 +0000 UTC m=+9.117407272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs") pod "network-metrics-daemon-8qf55" (UID: "ffad911d-9cef-4136-aeb5-ed9c6352add7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:46.312365 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:46.312061    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnl56\" (UniqueName: \"kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56\") pod \"network-check-target-tqlkj\" (UID: \"88d95652-29a9-4d30-b0e0-4795dc5e5077\") " pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:46.320371 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.320344    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:30:46.320852 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.320830    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:30:46.321088 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.321074    2549 projected.go:196] Error preparing data for projected volume kube-api-access-qnl56 for pod openshift-network-diagnostics/network-check-target-tqlkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:46.322170 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.322137    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56 podName:88d95652-29a9-4d30-b0e0-4795dc5e5077 nodeName:}" failed. No retries permitted until 2026-09-10 13:30:50.321337339 +0000 UTC m=+9.227895364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qnl56" (UniqueName: "kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56") pod "network-check-target-tqlkj" (UID: "88d95652-29a9-4d30-b0e0-4795dc5e5077") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:46.669765 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:46.668867    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:46.669765 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.669004    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:30:46.669765 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:46.669428    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:46.669765 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.669523    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:30:46.669765 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:46.669585    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:46.669765 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:46.669672    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:30:48.668570 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:48.668479    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:48.668570 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:48.668500    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:48.668570 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:48.668511    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:48.669133 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:48.668617    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:30:48.669133 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:48.668724    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:30:48.669133 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:48.668808    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:30:50.245417 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:50.245389    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:50.245417 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:50.245432    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:50.245931 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.245600    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:50.245931 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.245649    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:50.245931 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.245666    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret podName:fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e nodeName:}" failed. No retries permitted until 2026-09-10 13:30:58.245647896 +0000 UTC m=+17.152205908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret") pod "global-pull-secret-syncer-7jhj2" (UID: "fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:50.245931 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.245713    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs podName:ffad911d-9cef-4136-aeb5-ed9c6352add7 nodeName:}" failed. No retries permitted until 2026-09-10 13:30:58.245695057 +0000 UTC m=+17.152253070 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs") pod "network-metrics-daemon-8qf55" (UID: "ffad911d-9cef-4136-aeb5-ed9c6352add7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:50.346658 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:50.346488    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnl56\" (UniqueName: \"kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56\") pod \"network-check-target-tqlkj\" (UID: \"88d95652-29a9-4d30-b0e0-4795dc5e5077\") " pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:50.346658 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.346643    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:30:50.346658 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.346670    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:30:50.346942 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.346683    2549 projected.go:196] Error preparing data for projected volume kube-api-access-qnl56 for pod openshift-network-diagnostics/network-check-target-tqlkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:50.346942 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.346747    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56 podName:88d95652-29a9-4d30-b0e0-4795dc5e5077 nodeName:}" failed. No retries permitted until 2026-09-10 13:30:58.346728169 +0000 UTC m=+17.253286458 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qnl56" (UniqueName: "kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56") pod "network-check-target-tqlkj" (UID: "88d95652-29a9-4d30-b0e0-4795dc5e5077") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:50.669793 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:50.668964    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:50.669793 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.669122    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:30:50.669793 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:50.669499    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:50.669793 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.669606    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:30:50.669793 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:50.669658    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:50.669793 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:50.669755    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:30:52.668133 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:52.668102    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:52.668548 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:52.668103    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:52.668548 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:52.668217    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:30:52.668548 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:52.668103    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:52.668548 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:52.668286    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:30:52.668548 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:52.668419    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:30:54.668411 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:54.668374    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:54.668411 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:54.668412    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:54.668888 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:54.668484    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:54.668888 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:54.668500    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:30:54.668888 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:54.668562    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:30:54.668888 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:54.668674    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:30:56.668847 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:56.668767    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:56.669282 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:56.668892    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:56.669282 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:56.668898    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:30:56.669282 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:56.668903    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:56.669282 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:56.669008    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:30:56.669282 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:56.669111    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:30:58.300932 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:58.300895    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:58.301394 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:58.300941    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:58.301394 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.301076    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:58.301394 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.301134    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:58.301394 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.301149    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs podName:ffad911d-9cef-4136-aeb5-ed9c6352add7 nodeName:}" failed. No retries permitted until 2026-09-10 13:31:14.30112824 +0000 UTC m=+33.207686266 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs") pod "network-metrics-daemon-8qf55" (UID: "ffad911d-9cef-4136-aeb5-ed9c6352add7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:30:58.301394 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.301182    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret podName:fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e nodeName:}" failed. No retries permitted until 2026-09-10 13:31:14.301166054 +0000 UTC m=+33.207724064 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret") pod "global-pull-secret-syncer-7jhj2" (UID: "fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:30:58.402018 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:58.401981    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnl56\" (UniqueName: \"kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56\") pod \"network-check-target-tqlkj\" (UID: \"88d95652-29a9-4d30-b0e0-4795dc5e5077\") " pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:58.402197 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.402165    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:30:58.402197 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.402194    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:30:58.402292 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.402208    2549 projected.go:196] Error preparing data for projected volume kube-api-access-qnl56 for pod openshift-network-diagnostics/network-check-target-tqlkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:58.402292 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.402259    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56 podName:88d95652-29a9-4d30-b0e0-4795dc5e5077 nodeName:}" failed. No retries permitted until 2026-09-10 13:31:14.402246229 +0000 UTC m=+33.308804250 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qnl56" (UniqueName: "kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56") pod "network-check-target-tqlkj" (UID: "88d95652-29a9-4d30-b0e0-4795dc5e5077") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:30:58.668239 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:58.668212    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:30:58.668469 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:58.668246    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:30:58.668469 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:30:58.668214    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:30:58.668469 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.668327    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:30:58.668469 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.668429    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:30:58.668640 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:30:58.668501    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:31:00.668767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:00.668731    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:00.669241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:00.668773    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:00.669241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:00.668773    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:00.669241 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:00.668847    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:31:00.669241 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:00.668992    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:31:00.669241 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:00.669088    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:31:02.668771 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.668618    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:02.669395 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.668618    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:02.669395 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:02.668850    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:31:02.669395 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.668639    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:02.669395 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:02.668986    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:31:02.669395 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:02.669014    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:31:02.776198 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.775974    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pw9wz" event={"ID":"dc277d0a-0d17-499c-96e8-50582c9a2655","Type":"ContainerStarted","Data":"8e518b7a74063aab49c94b21bfe4691a268d4208652d24c4bfc5b404615d143a"}
Sep 10 13:31:02.779099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.779076    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:31:02.779428 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.779406    2549 generic.go:356] "Generic (PLEG): container finished" podID="96a4f241-7d7a-45e1-b838-e85ce3b4dad1" containerID="eb398dde927a995968188a69f99d828f515432377993c5444258bb82a581f998" exitCode=1
Sep 10 13:31:02.779490 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.779442    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" event={"ID":"96a4f241-7d7a-45e1-b838-e85ce3b4dad1","Type":"ContainerStarted","Data":"c916be3b5d21c4a539c150979a687c1b1b6f0492711767ef90a5a6952a913512"}
Sep 10 13:31:02.779547 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.779522    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" event={"ID":"96a4f241-7d7a-45e1-b838-e85ce3b4dad1","Type":"ContainerStarted","Data":"36ce2e883f581c1dc8c8a8420443f00eedeeb4475dc6b7cac811964ac00d6368"}
Sep 10 13:31:02.779590 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.779561    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" event={"ID":"96a4f241-7d7a-45e1-b838-e85ce3b4dad1","Type":"ContainerStarted","Data":"d20c7a9b9170024c6ab17774b4c73c9f3baa17b0d837a06332a58e6df14ee5fc"}
Sep 10 13:31:02.779639 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.779606    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" event={"ID":"96a4f241-7d7a-45e1-b838-e85ce3b4dad1","Type":"ContainerStarted","Data":"6fb422c02dcd9d8836e704a9d62baf11f282de8bc9ad9cd08bd743ec02a24fc6"}
Sep 10 13:31:02.779639 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.779636    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" event={"ID":"96a4f241-7d7a-45e1-b838-e85ce3b4dad1","Type":"ContainerDied","Data":"eb398dde927a995968188a69f99d828f515432377993c5444258bb82a581f998"}
Sep 10 13:31:02.779737 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.779664    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" event={"ID":"96a4f241-7d7a-45e1-b838-e85ce3b4dad1","Type":"ContainerStarted","Data":"e330340de82cbfdb172fa5204c6623819f158af70c068499aa46072e7cf841ed"}
Sep 10 13:31:02.781198 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.781175    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal" event={"ID":"651cc6668ff972ae51b87e7dde9a4f79","Type":"ContainerStarted","Data":"88be47ebc196edafca6062e401cbe17d62b1e74f0cfdcf3ca37fb4d0f5429977"}
Sep 10 13:31:02.782807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.782790    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nzhpv" event={"ID":"46769783-6500-4cc8-8447-587bed957524","Type":"ContainerStarted","Data":"608ab77da19f9c4f61283bb4ce2f7bb9bb0cff566f5792a76a40b226a32301a3"}
Sep 10 13:31:02.784204 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.784186    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wqzv5" event={"ID":"3986726d-5c3a-4a89-a469-58d7d779f0a5","Type":"ContainerStarted","Data":"488821823bec012384b94e3117c4d1328ca9833b021cca7d6229fbe08f785bc0"}
Sep 10 13:31:02.785681 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.785659    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dthlr" event={"ID":"36a92305-52ee-42ba-b9f4-12f6633781ef","Type":"ContainerStarted","Data":"584f39e45634af30801fbff71b6235ed79dc4bbf6843b2bad3dfa04f24c87ade"}
Sep 10 13:31:02.794695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.794660    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pw9wz" podStartSLOduration=1.928668916 podStartE2EDuration="20.794649113s" totalImagesPullingTime="18.865980197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:02.794275102 +0000 UTC m=+21.700833132" watchObservedRunningTime="2026-09-10 13:31:02.794649113 +0000 UTC m=+21.701207142"
Sep 10 13:31:02.818379 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.816767    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wqzv5" podStartSLOduration=1.472204455 podStartE2EDuration="20.816716156s" totalImagesPullingTime="19.344511701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:02.815490715 +0000 UTC m=+21.722048745" watchObservedRunningTime="2026-09-10 13:31:02.816716156 +0000 UTC m=+21.723274186"
Sep 10 13:31:02.840350 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.840304    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nzhpv" podStartSLOduration=2.013794093 podStartE2EDuration="20.84029129s" totalImagesPullingTime="18.826497197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:02.840269313 +0000 UTC m=+21.746827344" watchObservedRunningTime="2026-09-10 13:31:02.84029129 +0000 UTC m=+21.746849319"
Sep 10 13:31:02.858320 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:02.858280    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal" podStartSLOduration=20.858269225 podStartE2EDuration="20.858269225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:02.858034437 +0000 UTC m=+21.764592520" watchObservedRunningTime="2026-09-10 13:31:02.858269225 +0000 UTC m=+21.764827253"
Sep 10 13:31:03.790342 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.790307    2549 generic.go:356] "Generic (PLEG): container finished" podID="cb832ba95c432514fb054233d143126e" containerID="90483d7ac19afd239228d416bafb8fd4bd4124c292d1bb82c1a5eeb8aa671873" exitCode=0
Sep 10 13:31:03.791025 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.790402    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal" event={"ID":"cb832ba95c432514fb054233d143126e","Type":"ContainerDied","Data":"90483d7ac19afd239228d416bafb8fd4bd4124c292d1bb82c1a5eeb8aa671873"}
Sep 10 13:31:03.792557 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.792534    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vkmn" event={"ID":"2e530ccd-d80f-4199-8b0d-5de767763253","Type":"ContainerStarted","Data":"a8b6614d0dcff8799b1ce0f4e35e18e1b27ed25b18e437d2a2416c87ce57b4e7"}
Sep 10 13:31:03.794717 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.794696    2549 generic.go:356] "Generic (PLEG): container finished" podID="36a92305-52ee-42ba-b9f4-12f6633781ef" containerID="584f39e45634af30801fbff71b6235ed79dc4bbf6843b2bad3dfa04f24c87ade" exitCode=0
Sep 10 13:31:03.794826 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.794758    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dthlr" event={"ID":"36a92305-52ee-42ba-b9f4-12f6633781ef","Type":"ContainerDied","Data":"584f39e45634af30801fbff71b6235ed79dc4bbf6843b2bad3dfa04f24c87ade"}
Sep 10 13:31:03.796350 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.796324    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc" event={"ID":"c3ef299f-589b-4956-ab88-113c8cb03f26","Type":"ContainerStarted","Data":"7ec01245e03ce61ecb7839c4063f820c43a605bda73d97337266d84debeaaa44"}
Sep 10 13:31:03.797911 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.797887    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tdx8g" event={"ID":"69d18837-83dd-4960-a093-eabc75e22ddb","Type":"ContainerStarted","Data":"8952ce9afa5fc097bae12189944d658f2efced9de35a22321b8f1af3bf86c0a2"}
Sep 10 13:31:03.799569 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.799541    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vrlf8" event={"ID":"558a818a-f2a0-4a6c-aa06-ec3683eab67b","Type":"ContainerStarted","Data":"52d4187ce97f3d1b0bc9b05b1365af95ed26b3569ec4b2ce49b2439b7952c49f"}
Sep 10 13:31:03.865526 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.864842    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vrlf8" podStartSLOduration=3.04591038 podStartE2EDuration="21.864787241s" totalImagesPullingTime="18.818876861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:03.832592862 +0000 UTC m=+22.739150944" watchObservedRunningTime="2026-09-10 13:31:03.864787241 +0000 UTC m=+22.771345271"
Sep 10 13:31:03.893320 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.893270    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7vkmn" podStartSLOduration=3.087564112 podStartE2EDuration="21.893251792s" totalImagesPullingTime="18.80568768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:03.865406701 +0000 UTC m=+22.771964745" watchObservedRunningTime="2026-09-10 13:31:03.893251792 +0000 UTC m=+22.799809826"
Sep 10 13:31:03.893652 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:03.893623    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tdx8g" podStartSLOduration=3.114769663 podStartE2EDuration="21.8936133s" totalImagesPullingTime="18.778843637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:03.893003211 +0000 UTC m=+22.799561243" watchObservedRunningTime="2026-09-10 13:31:03.8936133 +0000 UTC m=+22.800171333"
Sep 10 13:31:04.231654 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.231620    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:31:04.232232 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.232210    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:31:04.374280 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.374261    2549 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 13:31:04.518480 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.518348    2549 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T13:31:04.374276358Z","UUID":"c5892171-4182-4377-ad60-1e53713a964d","Handler":null,"Name":"","Endpoint":""}
Sep 10 13:31:04.519713 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.519695    2549 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 13:31:04.519807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.519735    2549 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 13:31:04.668314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.668032    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:04.668314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.668047    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:04.668314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.668047    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:04.668314 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:04.668142    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:31:04.668314 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:04.668223    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:31:04.668314 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:04.668296    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:31:04.805782 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.805711    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:31:04.806423 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.806119    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" event={"ID":"96a4f241-7d7a-45e1-b838-e85ce3b4dad1","Type":"ContainerStarted","Data":"57e4ef20e922483fa1a66e95d28a8212249443f52b75f3bca1869e593e11289c"}
Sep 10 13:31:04.808034 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.808004    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal" event={"ID":"cb832ba95c432514fb054233d143126e","Type":"ContainerStarted","Data":"0e92b73f80e8579e3f13857e629e0c2271f7cee5e163de9b58d973dabff2f83f"}
Sep 10 13:31:04.810065 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.810042    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc" event={"ID":"c3ef299f-589b-4956-ab88-113c8cb03f26","Type":"ContainerStarted","Data":"a77f412aa56af3717276bfc48542b90d6ede67d58492499a42a4fb9f5cc8054e"}
Sep 10 13:31:04.833355 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:04.833309    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal" podStartSLOduration=22.833298822 podStartE2EDuration="22.833298822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:04.833114657 +0000 UTC m=+23.739672687" watchObservedRunningTime="2026-09-10 13:31:04.833298822 +0000 UTC m=+23.739856852"
Sep 10 13:31:05.815075 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:05.814810    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc" event={"ID":"c3ef299f-589b-4956-ab88-113c8cb03f26","Type":"ContainerStarted","Data":"77c552b694c5d93f0c24405fb2a5da6015bb3672d1da777ae12610de97f4aa28"}
Sep 10 13:31:05.815075 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:05.814942    2549 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 13:31:05.848704 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:05.848661    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m8xtc" podStartSLOduration=2.598305459 podStartE2EDuration="23.848647319s" totalImagesPullingTime="21.25034186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:05.848297395 +0000 UTC m=+24.754855439" watchObservedRunningTime="2026-09-10 13:31:05.848647319 +0000 UTC m=+24.755205346"
Sep 10 13:31:06.668364 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:06.668336    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:06.668489 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:06.668446    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:06.668489 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:06.668451    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:31:06.668489 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:06.668481    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:06.668626 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:06.668570    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:31:06.668684 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:06.668665    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:31:08.669027 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:08.668854    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:08.669390 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:08.668883    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:08.669390 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:08.669103    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:31:08.669390 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:08.668883    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:08.669390 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:08.669177    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:31:08.669390 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:08.669284    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:31:08.825482 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:08.825456    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:31:08.825857 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:08.825834    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" event={"ID":"96a4f241-7d7a-45e1-b838-e85ce3b4dad1","Type":"ContainerStarted","Data":"d9dcb52a53cc23c7af810dab4d2c475895653375b439de14764bd1d3bfa572ba"}
Sep 10 13:31:08.826114 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:08.826096    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:31:08.826300 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:08.826279    2549 scope.go:117] "RemoveContainer" containerID="eb398dde927a995968188a69f99d828f515432377993c5444258bb82a581f998"
Sep 10 13:31:08.827831 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:08.827809    2549 generic.go:356] "Generic (PLEG): container finished" podID="36a92305-52ee-42ba-b9f4-12f6633781ef" containerID="c5aa8c2c2fc80f384043565cef2a0bf49d12b07e81a9a74cc1713a20efc228bf" exitCode=0
Sep 10 13:31:08.827932 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:08.827839    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dthlr" event={"ID":"36a92305-52ee-42ba-b9f4-12f6633781ef","Type":"ContainerDied","Data":"c5aa8c2c2fc80f384043565cef2a0bf49d12b07e81a9a74cc1713a20efc228bf"}
Sep 10 13:31:08.841759 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:08.841742    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:31:09.833655 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:09.833631    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:31:09.834205 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:09.834055    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" event={"ID":"96a4f241-7d7a-45e1-b838-e85ce3b4dad1","Type":"ContainerStarted","Data":"98d310d4fe9e916459c4464107fcf92ec3cb7d21a24731169a7988f7c2bebd7f"}
Sep 10 13:31:09.834332 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:09.834303    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:31:09.834440 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:09.834395    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:31:09.849907 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:09.849885    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:31:09.898963 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:09.898920    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8" podStartSLOduration=8.965357321 podStartE2EDuration="27.898906112s" totalImagesPullingTime="18.933548791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:09.897525447 +0000 UTC m=+28.804083478" watchObservedRunningTime="2026-09-10 13:31:09.898906112 +0000 UTC m=+28.805464151"
Sep 10 13:31:10.055257 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:10.055234    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8qf55"]
Sep 10 13:31:10.055356 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:10.055344    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:10.055449 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:10.055433    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:31:10.059736 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:10.059712    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tqlkj"]
Sep 10 13:31:10.059842 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:10.059802    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:10.059923 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:10.059888    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:31:10.060349 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:10.060328    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7jhj2"]
Sep 10 13:31:10.060431 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:10.060420    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:10.060520 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:10.060494    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:31:10.838039 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:10.838008    2549 generic.go:356] "Generic (PLEG): container finished" podID="36a92305-52ee-42ba-b9f4-12f6633781ef" containerID="d2c34fa2bf3456b4845f6cc5660ee770caa39ab3fd93374e88567e403a62dcdc" exitCode=0
Sep 10 13:31:10.838498 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:10.838086    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dthlr" event={"ID":"36a92305-52ee-42ba-b9f4-12f6633781ef","Type":"ContainerDied","Data":"d2c34fa2bf3456b4845f6cc5660ee770caa39ab3fd93374e88567e403a62dcdc"}
Sep 10 13:31:11.368405 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:11.368372    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:31:11.368560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:11.368548    2549 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 13:31:11.369027 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:11.369007    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tdx8g"
Sep 10 13:31:11.668755 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:11.668733    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:11.668911 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:11.668828    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:31:11.668911 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:11.668903    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:11.669005 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:11.668966    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:31:11.669052 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:11.669024    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:11.669153 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:11.669134    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:31:13.025729 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:13.025654    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pw9wz_dc277d0a-0d17-499c-96e8-50582c9a2655/dns-node-resolver/0.log"
Sep 10 13:31:13.668161 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:13.668133    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:13.668268 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:13.668141    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:13.668268 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:13.668234    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qf55" podUID="ffad911d-9cef-4136-aeb5-ed9c6352add7"
Sep 10 13:31:13.668352 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:13.668141    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:13.668352 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:13.668308    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jhj2" podUID="fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e"
Sep 10 13:31:13.668410 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:13.668394    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqlkj" podUID="88d95652-29a9-4d30-b0e0-4795dc5e5077"
Sep 10 13:31:13.805681 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:13.805657    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vkmn_2e530ccd-d80f-4199-8b0d-5de767763253/node-ca/0.log"
Sep 10 13:31:13.846773 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:13.846749    2549 generic.go:356] "Generic (PLEG): container finished" podID="36a92305-52ee-42ba-b9f4-12f6633781ef" containerID="10a45072d3a4b97cefb2b2aec7a5205f944e4e3e32092e2364584d7f3f3a3757" exitCode=0
Sep 10 13:31:13.846909 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:13.846800    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dthlr" event={"ID":"36a92305-52ee-42ba-b9f4-12f6633781ef","Type":"ContainerDied","Data":"10a45072d3a4b97cefb2b2aec7a5205f944e4e3e32092e2364584d7f3f3a3757"}
Sep 10 13:31:14.312134 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:14.312107    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeReady"
Sep 10 13:31:14.312510 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:14.312235    2549 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 13:31:14.317770 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:14.317747    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:14.317878 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:14.317827    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:14.317932 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:14.317892    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:31:14.317988 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:14.317944    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret podName:fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e nodeName:}" failed. No retries permitted until 2026-09-10 13:31:46.31793041 +0000 UTC m=+65.224488423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret") pod "global-pull-secret-syncer-7jhj2" (UID: "fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:31:14.317988 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:14.317942    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:31:14.317988 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:14.317969    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs podName:ffad911d-9cef-4136-aeb5-ed9c6352add7 nodeName:}" failed. No retries permitted until 2026-09-10 13:31:46.31796324 +0000 UTC m=+65.224521248 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs") pod "network-metrics-daemon-8qf55" (UID: "ffad911d-9cef-4136-aeb5-ed9c6352add7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:31:14.418750 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:14.418712    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnl56\" (UniqueName: \"kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56\") pod \"network-check-target-tqlkj\" (UID: \"88d95652-29a9-4d30-b0e0-4795dc5e5077\") " pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:14.418893 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:14.418839    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:31:14.418893 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:14.418856    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:31:14.418893 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:14.418882    2549 projected.go:196] Error preparing data for projected volume kube-api-access-qnl56 for pod openshift-network-diagnostics/network-check-target-tqlkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:31:14.419016 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:14.418933    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56 podName:88d95652-29a9-4d30-b0e0-4795dc5e5077 nodeName:}" failed. No retries permitted until 2026-09-10 13:31:46.4189193 +0000 UTC m=+65.325477308 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qnl56" (UniqueName: "kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56") pod "network-check-target-tqlkj" (UID: "88d95652-29a9-4d30-b0e0-4795dc5e5077") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:31:15.668528 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:15.668494    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:15.668528 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:15.668527    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:15.669189 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:15.668667    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:15.672645 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:15.672626    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 13:31:15.673362 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:15.673340    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 13:31:15.673774 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:15.673742    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 13:31:15.673774 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:15.673755    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nc4jc\""
Sep 10 13:31:15.673774 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:15.673747    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 13:31:15.673986 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:15.673826    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-twwrf\""
Sep 10 13:31:19.862634 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:19.862561    2549 generic.go:356] "Generic (PLEG): container finished" podID="36a92305-52ee-42ba-b9f4-12f6633781ef" containerID="d41332413d76c0854fa95f29ed69f5f1bae9a136d5984e627fdbcfdaa1231ada" exitCode=0
Sep 10 13:31:19.862634 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:19.862627    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dthlr" event={"ID":"36a92305-52ee-42ba-b9f4-12f6633781ef","Type":"ContainerDied","Data":"d41332413d76c0854fa95f29ed69f5f1bae9a136d5984e627fdbcfdaa1231ada"}
Sep 10 13:31:20.868316 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:20.868142    2549 generic.go:356] "Generic (PLEG): container finished" podID="36a92305-52ee-42ba-b9f4-12f6633781ef" containerID="c8731d24ffe2463ba3bc55d0a9b00529608b10c38ccfb12f1da12fe5d3670c73" exitCode=0
Sep 10 13:31:20.868316 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:20.868223    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dthlr" event={"ID":"36a92305-52ee-42ba-b9f4-12f6633781ef","Type":"ContainerDied","Data":"c8731d24ffe2463ba3bc55d0a9b00529608b10c38ccfb12f1da12fe5d3670c73"}
Sep 10 13:31:21.873992 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:21.873950    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dthlr" event={"ID":"36a92305-52ee-42ba-b9f4-12f6633781ef","Type":"ContainerStarted","Data":"8e1bcf36705f41ac4f8f02ee52d96d97f6f9b5320f571ca1ff0bc3a23edc01ac"}
Sep 10 13:31:21.906990 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:21.906937    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dthlr" podStartSLOduration=7.192481243 podStartE2EDuration="39.906923065s" totalImagesPullingTime="32.714441822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:21.90576533 +0000 UTC m=+40.812323361" watchObservedRunningTime="2026-09-10 13:31:21.906923065 +0000 UTC m=+40.813481106"
Sep 10 13:31:34.381967 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.381931    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-666f7c8b58-fnnn9"]
Sep 10 13:31:34.383608 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.383592    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.386487 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.386446    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 13:31:34.387064 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.386731    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tn4wt\""
Sep 10 13:31:34.387432 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.387405    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 13:31:34.387522 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.387438    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 13:31:34.391657 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.391635    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 13:31:34.403961 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.403936    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-666f7c8b58-fnnn9"]
Sep 10 13:31:34.419751 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.419727    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-666f7c8b58-fnnn9"]
Sep 10 13:31:34.419953 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:31:34.419931    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-dnsz9 registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-dnsz9 registry-certificates registry-tls trusted-ca]: context canceled" pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9" podUID="c924325f-23fe-44fa-b6d7-2f219b989ea8"
Sep 10 13:31:34.450991 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.450962    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"]
Sep 10 13:31:34.452662 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.452644    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"]
Sep 10 13:31:34.452796 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.452776    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.454331 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.454311    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"
Sep 10 13:31:34.455379 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.455361    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"]
Sep 10 13:31:34.455502 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.455484    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 13:31:34.455812 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.455791    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 13:31:34.455913 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.455792    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 13:31:34.455913 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.455791    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 13:31:34.457163 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.457147    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.457532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.457518    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 13:31:34.457622 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.457607    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-g4bd2\""
Sep 10 13:31:34.459934 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.459915    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 13:31:34.460033 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.459941    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 13:31:34.460033 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.459943    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 13:31:34.460112 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.460044    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 13:31:34.484370 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.484118    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"]
Sep 10 13:31:34.484660 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.484608    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"]
Sep 10 13:31:34.485313 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.485292    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"]
Sep 10 13:31:34.539982 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.539952    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c947ff77-zhw2x"]
Sep 10 13:31:34.541693 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.541678    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.544152 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.544123    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dbz76"]
Sep 10 13:31:34.545700 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.545686    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5wjmf"]
Sep 10 13:31:34.545806 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.545792    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.547473 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547456    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.547570 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547553    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-hub\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.547652 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547633    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnsz9\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-kube-api-access-dnsz9\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.547704 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547666    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b8fa133-e000-42cb-af84-2198d386693b-tmp\") pod \"klusterlet-addon-workmgr-5d87c76d54-vw5q7\" (UID: \"2b8fa133-e000-42cb-af84-2198d386693b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.547747 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547717    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.547779 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547757    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-bound-sa-token\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.547811 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547779    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgfkh\" (UniqueName: \"kubernetes.io/projected/a162239c-7105-4acb-a515-7b5abde239da-kube-api-access-kgfkh\") pod \"managed-serviceaccount-addon-agent-7695494676-5q6jt\" (UID: \"a162239c-7105-4acb-a515-7b5abde239da\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"
Sep 10 13:31:34.547811 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547796    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-certificates\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.547889 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547837    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.547889 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547859    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-ca\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.547956 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547892    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2b8fa133-e000-42cb-af84-2198d386693b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d87c76d54-vw5q7\" (UID: \"2b8fa133-e000-42cb-af84-2198d386693b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.547956 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547921    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c924325f-23fe-44fa-b6d7-2f219b989ea8-ca-trust-extracted\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.548016 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547958    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx28x\" (UniqueName: \"kubernetes.io/projected/2b8fa133-e000-42cb-af84-2198d386693b-kube-api-access-vx28x\") pod \"klusterlet-addon-workmgr-5d87c76d54-vw5q7\" (UID: \"2b8fa133-e000-42cb-af84-2198d386693b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.548016 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.547993    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlvfk\" (UniqueName: \"kubernetes.io/projected/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-kube-api-access-xlvfk\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.548076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.548023    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-tls\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.548076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.548040    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a162239c-7105-4acb-a515-7b5abde239da-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7695494676-5q6jt\" (UID: \"a162239c-7105-4acb-a515-7b5abde239da\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"
Sep 10 13:31:34.548076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.548054    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.548168 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.548095    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-trusted-ca\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.548168 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.548111    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-installation-pull-secrets\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.548168 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.548142    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-image-registry-private-configuration\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.548289 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.548276    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 13:31:34.548417 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.548403    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p9764\""
Sep 10 13:31:34.548484 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.548409    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 13:31:34.549710 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.549696    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 13:31:34.549754 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.549696    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 13:31:34.549900 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.549857    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2d75h\""
Sep 10 13:31:34.550116 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.550104    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 13:31:34.550158 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.550109    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 13:31:34.564281 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.564254    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c947ff77-zhw2x"]
Sep 10 13:31:34.564907 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.564887    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dbz76"]
Sep 10 13:31:34.566704 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.566681    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5wjmf"]
Sep 10 13:31:34.641521 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.641447    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bhpvl"]
Sep 10 13:31:34.643422 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.643407    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bhpvl"
Sep 10 13:31:34.645784 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.645761    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w88gf\""
Sep 10 13:31:34.646077 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.646052    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 13:31:34.646173 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.646090    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 13:31:34.646689 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.646672    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 13:31:34.648655 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648629    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-trusted-ca\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.648765 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648671    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b42d2757-0d53-4232-8a6a-2039f0b93e0f-crio-socket\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.648765 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648705    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-hub\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.648765 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648732    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-registry-certificates\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.648765 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648759    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-registry-tls\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.649008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648829    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-ca-trust-extracted\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.649008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648892    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.649008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648915    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-bound-sa-token\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.649008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648931    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2b8fa133-e000-42cb-af84-2198d386693b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d87c76d54-vw5q7\" (UID: \"2b8fa133-e000-42cb-af84-2198d386693b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.649008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648950    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-bound-sa-token\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.649008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.648974    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b42d2757-0d53-4232-8a6a-2039f0b93e0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.649008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649001    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgfkh\" (UniqueName: \"kubernetes.io/projected/a162239c-7105-4acb-a515-7b5abde239da-kube-api-access-kgfkh\") pod \"managed-serviceaccount-addon-agent-7695494676-5q6jt\" (UID: \"a162239c-7105-4acb-a515-7b5abde239da\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"
Sep 10 13:31:34.649336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649025    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.649336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649073    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.649336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649100    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-installation-pull-secrets\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.649336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649150    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-installation-pull-secrets\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.649336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649175    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b42d2757-0d53-4232-8a6a-2039f0b93e0f-data-volume\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.649336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649216    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnsz9\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-kube-api-access-dnsz9\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.649336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649243    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-image-registry-private-configuration\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.649336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649268    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94abed11-8633-43cc-98a0-19223098fa40-tmp-dir\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.649336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649295    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b8fa133-e000-42cb-af84-2198d386693b-tmp\") pod \"klusterlet-addon-workmgr-5d87c76d54-vw5q7\" (UID: \"2b8fa133-e000-42cb-af84-2198d386693b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.649336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649320    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-image-registry-private-configuration\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649362    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4rj8\" (UniqueName: \"kubernetes.io/projected/94abed11-8633-43cc-98a0-19223098fa40-kube-api-access-l4rj8\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649394    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84pwd\" (UniqueName: \"kubernetes.io/projected/b42d2757-0d53-4232-8a6a-2039f0b93e0f-kube-api-access-84pwd\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649422    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-certificates\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649450    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-trusted-ca\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649475    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b42d2757-0d53-4232-8a6a-2039f0b93e0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649505    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-ca\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649532    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfld8\" (UniqueName: \"kubernetes.io/projected/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-kube-api-access-rfld8\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649572    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c924325f-23fe-44fa-b6d7-2f219b989ea8-ca-trust-extracted\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649599    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx28x\" (UniqueName: \"kubernetes.io/projected/2b8fa133-e000-42cb-af84-2198d386693b-kube-api-access-vx28x\") pod \"klusterlet-addon-workmgr-5d87c76d54-vw5q7\" (UID: \"2b8fa133-e000-42cb-af84-2198d386693b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649625    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlvfk\" (UniqueName: \"kubernetes.io/projected/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-kube-api-access-xlvfk\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649628    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-trusted-ca\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649650    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94abed11-8633-43cc-98a0-19223098fa40-metrics-tls\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649665    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649684    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-tls\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.649807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649713    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a162239c-7105-4acb-a515-7b5abde239da-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7695494676-5q6jt\" (UID: \"a162239c-7105-4acb-a515-7b5abde239da\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"
Sep 10 13:31:34.650645 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.649738    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94abed11-8633-43cc-98a0-19223098fa40-config-volume\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.650645 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.650327    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-certificates\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.650817 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.650797    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c924325f-23fe-44fa-b6d7-2f219b989ea8-ca-trust-extracted\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.650946 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.650928    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b8fa133-e000-42cb-af84-2198d386693b-tmp\") pod \"klusterlet-addon-workmgr-5d87c76d54-vw5q7\" (UID: \"2b8fa133-e000-42cb-af84-2198d386693b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.654236 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.654174    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-tls\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.654335 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.654267    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2b8fa133-e000-42cb-af84-2198d386693b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d87c76d54-vw5q7\" (UID: \"2b8fa133-e000-42cb-af84-2198d386693b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.654335 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.654277    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.654446 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.654355    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-hub\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.654633 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.654601    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-ca\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.654704 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.654683    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a162239c-7105-4acb-a515-7b5abde239da-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7695494676-5q6jt\" (UID: \"a162239c-7105-4acb-a515-7b5abde239da\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"
Sep 10 13:31:34.655164 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.655147    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-image-registry-private-configuration\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.655479 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.655457    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-installation-pull-secrets\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.655579 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.655564    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.663167 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.663137    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bhpvl"]
Sep 10 13:31:34.669315 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.669290    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx28x\" (UniqueName: \"kubernetes.io/projected/2b8fa133-e000-42cb-af84-2198d386693b-kube-api-access-vx28x\") pod \"klusterlet-addon-workmgr-5d87c76d54-vw5q7\" (UID: \"2b8fa133-e000-42cb-af84-2198d386693b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.669497 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.669482    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-bound-sa-token\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.669537 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.669510    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlvfk\" (UniqueName: \"kubernetes.io/projected/4ac14fc2-1f56-4e75-ab8f-04e54df2848e-kube-api-access-xlvfk\") pod \"cluster-proxy-proxy-agent-7d9f96db6b-gqs5s\" (UID: \"4ac14fc2-1f56-4e75-ab8f-04e54df2848e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.669655 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.669638    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgfkh\" (UniqueName: \"kubernetes.io/projected/a162239c-7105-4acb-a515-7b5abde239da-kube-api-access-kgfkh\") pod \"managed-serviceaccount-addon-agent-7695494676-5q6jt\" (UID: \"a162239c-7105-4acb-a515-7b5abde239da\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"
Sep 10 13:31:34.670926 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.670910    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnsz9\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-kube-api-access-dnsz9\") pod \"image-registry-666f7c8b58-fnnn9\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") " pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.750314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750282    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94abed11-8633-43cc-98a0-19223098fa40-metrics-tls\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.750314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750316    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94abed11-8633-43cc-98a0-19223098fa40-config-volume\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.750508 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750338    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b42d2757-0d53-4232-8a6a-2039f0b93e0f-crio-socket\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.750508 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750367    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-registry-certificates\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.750508 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750390    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-registry-tls\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.750508 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750406    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-ca-trust-extracted\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.750508 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750422    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-bound-sa-token\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.750508 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750438    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b42d2757-0d53-4232-8a6a-2039f0b93e0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.750508 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750474    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-installation-pull-secrets\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.750508 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750510    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gg4t\" (UniqueName: \"kubernetes.io/projected/c1f29aa0-318e-4f86-8500-4e3b68b848f7-kube-api-access-2gg4t\") pod \"ingress-canary-bhpvl\" (UID: \"c1f29aa0-318e-4f86-8500-4e3b68b848f7\") " pod="openshift-ingress-canary/ingress-canary-bhpvl"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750531    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b42d2757-0d53-4232-8a6a-2039f0b93e0f-data-volume\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750546    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b42d2757-0d53-4232-8a6a-2039f0b93e0f-crio-socket\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750570    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94abed11-8633-43cc-98a0-19223098fa40-tmp-dir\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750596    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1f29aa0-318e-4f86-8500-4e3b68b848f7-cert\") pod \"ingress-canary-bhpvl\" (UID: \"c1f29aa0-318e-4f86-8500-4e3b68b848f7\") " pod="openshift-ingress-canary/ingress-canary-bhpvl"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750630    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-image-registry-private-configuration\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750667    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4rj8\" (UniqueName: \"kubernetes.io/projected/94abed11-8633-43cc-98a0-19223098fa40-kube-api-access-l4rj8\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750698    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84pwd\" (UniqueName: \"kubernetes.io/projected/b42d2757-0d53-4232-8a6a-2039f0b93e0f-kube-api-access-84pwd\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750729    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-trusted-ca\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750751    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b42d2757-0d53-4232-8a6a-2039f0b93e0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750778    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfld8\" (UniqueName: \"kubernetes.io/projected/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-kube-api-access-rfld8\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.750895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.750826    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-ca-trust-extracted\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.751412 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.751018    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b42d2757-0d53-4232-8a6a-2039f0b93e0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.751412 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.751009    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94abed11-8633-43cc-98a0-19223098fa40-config-volume\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.751412 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.751279    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b42d2757-0d53-4232-8a6a-2039f0b93e0f-data-volume\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.751412 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.751312    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-registry-certificates\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.751600 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.751569    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94abed11-8633-43cc-98a0-19223098fa40-tmp-dir\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.751960 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.751932    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-trusted-ca\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.752989 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.752969    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94abed11-8633-43cc-98a0-19223098fa40-metrics-tls\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.753323 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.753303    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-installation-pull-secrets\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.753440 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.753419    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b42d2757-0d53-4232-8a6a-2039f0b93e0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.753673 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.753654    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-registry-tls\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.753722 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.753708    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-image-registry-private-configuration\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.762654 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.762638    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:34.773018 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.772993    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfld8\" (UniqueName: \"kubernetes.io/projected/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-kube-api-access-rfld8\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.773018 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.773002    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4rj8\" (UniqueName: \"kubernetes.io/projected/94abed11-8633-43cc-98a0-19223098fa40-kube-api-access-l4rj8\") pod \"dns-default-dbz76\" (UID: \"94abed11-8633-43cc-98a0-19223098fa40\") " pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.773177 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.773153    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84pwd\" (UniqueName: \"kubernetes.io/projected/b42d2757-0d53-4232-8a6a-2039f0b93e0f-kube-api-access-84pwd\") pod \"insights-runtime-extractor-5wjmf\" (UID: \"b42d2757-0d53-4232-8a6a-2039f0b93e0f\") " pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.774192 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.774171    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d578d4a-e2a6-46c5-b1eb-6384a3509f2b-bound-sa-token\") pod \"image-registry-6c947ff77-zhw2x\" (UID: \"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b\") " pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.779952 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.779936    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"
Sep 10 13:31:34.786873 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.786844    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"
Sep 10 13:31:34.852610 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.852047    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gg4t\" (UniqueName: \"kubernetes.io/projected/c1f29aa0-318e-4f86-8500-4e3b68b848f7-kube-api-access-2gg4t\") pod \"ingress-canary-bhpvl\" (UID: \"c1f29aa0-318e-4f86-8500-4e3b68b848f7\") " pod="openshift-ingress-canary/ingress-canary-bhpvl"
Sep 10 13:31:34.852610 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.852103    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1f29aa0-318e-4f86-8500-4e3b68b848f7-cert\") pod \"ingress-canary-bhpvl\" (UID: \"c1f29aa0-318e-4f86-8500-4e3b68b848f7\") " pod="openshift-ingress-canary/ingress-canary-bhpvl"
Sep 10 13:31:34.856404 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.856382    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1f29aa0-318e-4f86-8500-4e3b68b848f7-cert\") pod \"ingress-canary-bhpvl\" (UID: \"c1f29aa0-318e-4f86-8500-4e3b68b848f7\") " pod="openshift-ingress-canary/ingress-canary-bhpvl"
Sep 10 13:31:34.866217 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.866196    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:34.873212 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.873194    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:34.878059 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.878037    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5wjmf"
Sep 10 13:31:34.881765 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.881737    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gg4t\" (UniqueName: \"kubernetes.io/projected/c1f29aa0-318e-4f86-8500-4e3b68b848f7-kube-api-access-2gg4t\") pod \"ingress-canary-bhpvl\" (UID: \"c1f29aa0-318e-4f86-8500-4e3b68b848f7\") " pod="openshift-ingress-canary/ingress-canary-bhpvl"
Sep 10 13:31:34.901565 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.901143    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.908110 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.908029    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:34.953890 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:34.953849    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bhpvl"
Sep 10 13:31:35.053392 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.053361    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-installation-pull-secrets\") pod \"c924325f-23fe-44fa-b6d7-2f219b989ea8\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") "
Sep 10 13:31:35.053392 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.053417    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-image-registry-private-configuration\") pod \"c924325f-23fe-44fa-b6d7-2f219b989ea8\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") "
Sep 10 13:31:35.053721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.053528    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-tls\") pod \"c924325f-23fe-44fa-b6d7-2f219b989ea8\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") "
Sep 10 13:31:35.053721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.053571    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c924325f-23fe-44fa-b6d7-2f219b989ea8-ca-trust-extracted\") pod \"c924325f-23fe-44fa-b6d7-2f219b989ea8\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") "
Sep 10 13:31:35.053721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.053603    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnsz9\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-kube-api-access-dnsz9\") pod \"c924325f-23fe-44fa-b6d7-2f219b989ea8\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") "
Sep 10 13:31:35.053721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.053630    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-bound-sa-token\") pod \"c924325f-23fe-44fa-b6d7-2f219b989ea8\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") "
Sep 10 13:31:35.053721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.053669    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-trusted-ca\") pod \"c924325f-23fe-44fa-b6d7-2f219b989ea8\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") "
Sep 10 13:31:35.053721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.053719    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-certificates\") pod \"c924325f-23fe-44fa-b6d7-2f219b989ea8\" (UID: \"c924325f-23fe-44fa-b6d7-2f219b989ea8\") "
Sep 10 13:31:35.054319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.054291    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c924325f-23fe-44fa-b6d7-2f219b989ea8" (UID: "c924325f-23fe-44fa-b6d7-2f219b989ea8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:31:35.054319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.054308    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c924325f-23fe-44fa-b6d7-2f219b989ea8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c924325f-23fe-44fa-b6d7-2f219b989ea8" (UID: "c924325f-23fe-44fa-b6d7-2f219b989ea8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:35.054602 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.054524    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"]
Sep 10 13:31:35.054721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.054660    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c924325f-23fe-44fa-b6d7-2f219b989ea8" (UID: "c924325f-23fe-44fa-b6d7-2f219b989ea8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:31:35.056311 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.056188    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c924325f-23fe-44fa-b6d7-2f219b989ea8" (UID: "c924325f-23fe-44fa-b6d7-2f219b989ea8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:31:35.056311 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.056235    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c924325f-23fe-44fa-b6d7-2f219b989ea8" (UID: "c924325f-23fe-44fa-b6d7-2f219b989ea8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:31:35.056467 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.056355    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c924325f-23fe-44fa-b6d7-2f219b989ea8" (UID: "c924325f-23fe-44fa-b6d7-2f219b989ea8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:31:35.056467 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.056406    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c924325f-23fe-44fa-b6d7-2f219b989ea8" (UID: "c924325f-23fe-44fa-b6d7-2f219b989ea8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:31:35.056667 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.056643    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-kube-api-access-dnsz9" (OuterVolumeSpecName: "kube-api-access-dnsz9") pod "c924325f-23fe-44fa-b6d7-2f219b989ea8" (UID: "c924325f-23fe-44fa-b6d7-2f219b989ea8"). InnerVolumeSpecName "kube-api-access-dnsz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:31:35.058252 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:35.058230    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b8fa133_e000_42cb_af84_2198d386693b.slice/crio-8b068f25cdf90cd4f27286f22b1a77374a1457876cc5278ab5c1c22e370ea473 WatchSource:0}: Error finding container 8b068f25cdf90cd4f27286f22b1a77374a1457876cc5278ab5c1c22e370ea473: Status 404 returned error can't find the container with id 8b068f25cdf90cd4f27286f22b1a77374a1457876cc5278ab5c1c22e370ea473
Sep 10 13:31:35.066771 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.066752    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt"]
Sep 10 13:31:35.071438 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:35.071416    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda162239c_7105_4acb_a515_7b5abde239da.slice/crio-bebe2d8299bd19ad699a28b7f588578ae2ebf33142461b4c68e0c5d5f8b4c479 WatchSource:0}: Error finding container bebe2d8299bd19ad699a28b7f588578ae2ebf33142461b4c68e0c5d5f8b4c479: Status 404 returned error can't find the container with id bebe2d8299bd19ad699a28b7f588578ae2ebf33142461b4c68e0c5d5f8b4c479
Sep 10 13:31:35.090761 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.088319    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s"]
Sep 10 13:31:35.093622 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:35.093597    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ac14fc2_1f56_4e75_ab8f_04e54df2848e.slice/crio-d49d9cd71bb102999fc01147bb5026113ffc7fa62b9edcdc82a85fd788221039 WatchSource:0}: Error finding container d49d9cd71bb102999fc01147bb5026113ffc7fa62b9edcdc82a85fd788221039: Status 404 returned error can't find the container with id d49d9cd71bb102999fc01147bb5026113ffc7fa62b9edcdc82a85fd788221039
Sep 10 13:31:35.152292 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.152226    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c947ff77-zhw2x"]
Sep 10 13:31:35.154590 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.154567    2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-trusted-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.154590 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.154589    2549 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-certificates\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.154736 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.154599    2549 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-installation-pull-secrets\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.154736 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.154616    2549 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c924325f-23fe-44fa-b6d7-2f219b989ea8-image-registry-private-configuration\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.154736 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.154629    2549 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-registry-tls\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.154736 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.154638    2549 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c924325f-23fe-44fa-b6d7-2f219b989ea8-ca-trust-extracted\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.154736 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.154645    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnsz9\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-kube-api-access-dnsz9\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.154736 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.154653    2549 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c924325f-23fe-44fa-b6d7-2f219b989ea8-bound-sa-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.155243 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:35.155214    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d578d4a_e2a6_46c5_b1eb_6384a3509f2b.slice/crio-92ef5962b25e4bd46bdd1cd3523709b6b759edaf59fde4ac043eb29f8ac44ad1 WatchSource:0}: Error finding container 92ef5962b25e4bd46bdd1cd3523709b6b759edaf59fde4ac043eb29f8ac44ad1: Status 404 returned error can't find the container with id 92ef5962b25e4bd46bdd1cd3523709b6b759edaf59fde4ac043eb29f8ac44ad1
Sep 10 13:31:35.172101 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.172081    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dbz76"]
Sep 10 13:31:35.175972 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.175954    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5wjmf"]
Sep 10 13:31:35.182913 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:35.182889    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94abed11_8633_43cc_98a0_19223098fa40.slice/crio-d0398db719c3769a386070c24c641f223608393ea517c0811fad750294e9a7e3 WatchSource:0}: Error finding container d0398db719c3769a386070c24c641f223608393ea517c0811fad750294e9a7e3: Status 404 returned error can't find the container with id d0398db719c3769a386070c24c641f223608393ea517c0811fad750294e9a7e3
Sep 10 13:31:35.183061 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:35.183046    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42d2757_0d53_4232_8a6a_2039f0b93e0f.slice/crio-7d89065fd7c18575312e5b91ad309e1ac2836e30696e5778e343b350623643bb WatchSource:0}: Error finding container 7d89065fd7c18575312e5b91ad309e1ac2836e30696e5778e343b350623643bb: Status 404 returned error can't find the container with id 7d89065fd7c18575312e5b91ad309e1ac2836e30696e5778e343b350623643bb
Sep 10 13:31:35.235349 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.235322    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bhpvl"]
Sep 10 13:31:35.240136 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:35.240109    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f29aa0_318e_4f86_8500_4e3b68b848f7.slice/crio-e8c214ef990ba4ab351b7b6af2b4a156609b3fc9f10310a82cad3cec36fe0388 WatchSource:0}: Error finding container e8c214ef990ba4ab351b7b6af2b4a156609b3fc9f10310a82cad3cec36fe0388: Status 404 returned error can't find the container with id e8c214ef990ba4ab351b7b6af2b4a156609b3fc9f10310a82cad3cec36fe0388
Sep 10 13:31:35.907954 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.907882    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt" event={"ID":"a162239c-7105-4acb-a515-7b5abde239da","Type":"ContainerStarted","Data":"bebe2d8299bd19ad699a28b7f588578ae2ebf33142461b4c68e0c5d5f8b4c479"}
Sep 10 13:31:35.909603 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.909504    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bhpvl" event={"ID":"c1f29aa0-318e-4f86-8500-4e3b68b848f7","Type":"ContainerStarted","Data":"e8c214ef990ba4ab351b7b6af2b4a156609b3fc9f10310a82cad3cec36fe0388"}
Sep 10 13:31:35.911606 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.911500    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5wjmf" event={"ID":"b42d2757-0d53-4232-8a6a-2039f0b93e0f","Type":"ContainerStarted","Data":"2f6f3d2f1260f1c5d209159d2b8642b785635a3b79d46a09e1e4c17d13471819"}
Sep 10 13:31:35.911701 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.911613    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5wjmf" event={"ID":"b42d2757-0d53-4232-8a6a-2039f0b93e0f","Type":"ContainerStarted","Data":"7d89065fd7c18575312e5b91ad309e1ac2836e30696e5778e343b350623643bb"}
Sep 10 13:31:35.914794 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.913984    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c947ff77-zhw2x" event={"ID":"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b","Type":"ContainerStarted","Data":"a16f1a247c00b32c824732ad6b5e19a680f5288e1b20a892d1807e653a7b6e01"}
Sep 10 13:31:35.914794 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.914021    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c947ff77-zhw2x" event={"ID":"3d578d4a-e2a6-46c5-b1eb-6384a3509f2b","Type":"ContainerStarted","Data":"92ef5962b25e4bd46bdd1cd3523709b6b759edaf59fde4ac043eb29f8ac44ad1"}
Sep 10 13:31:35.914794 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.914733    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:31:35.916639 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.916576    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s" event={"ID":"4ac14fc2-1f56-4e75-ab8f-04e54df2848e","Type":"ContainerStarted","Data":"d49d9cd71bb102999fc01147bb5026113ffc7fa62b9edcdc82a85fd788221039"}
Sep 10 13:31:35.917992 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.917928    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7" event={"ID":"2b8fa133-e000-42cb-af84-2198d386693b","Type":"ContainerStarted","Data":"8b068f25cdf90cd4f27286f22b1a77374a1457876cc5278ab5c1c22e370ea473"}
Sep 10 13:31:35.922033 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.921499    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666f7c8b58-fnnn9"
Sep 10 13:31:35.922033 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.921969    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dbz76" event={"ID":"94abed11-8633-43cc-98a0-19223098fa40","Type":"ContainerStarted","Data":"d0398db719c3769a386070c24c641f223608393ea517c0811fad750294e9a7e3"}
Sep 10 13:31:35.950350 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.950242    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c947ff77-zhw2x" podStartSLOduration=1.950221146 podStartE2EDuration="1.950221146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:35.949571052 +0000 UTC m=+54.856129191" watchObservedRunningTime="2026-09-10 13:31:35.950221146 +0000 UTC m=+54.856779176"
Sep 10 13:31:35.992354 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.992294    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-666f7c8b58-fnnn9"]
Sep 10 13:31:35.997086 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:35.997056    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-666f7c8b58-fnnn9"]
Sep 10 13:31:36.931595 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:36.931560    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5wjmf" event={"ID":"b42d2757-0d53-4232-8a6a-2039f0b93e0f","Type":"ContainerStarted","Data":"cfadd3cec6665b14adf573ff4933ea0e51557e91cac8dd172c81d1cb369073d7"}
Sep 10 13:31:37.673680 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:37.673465    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c924325f-23fe-44fa-b6d7-2f219b989ea8" path="/var/lib/kubelet/pods/c924325f-23fe-44fa-b6d7-2f219b989ea8/volumes"
Sep 10 13:31:41.851560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:41.851528    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b7sv8"
Sep 10 13:31:43.955094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.955050    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt" event={"ID":"a162239c-7105-4acb-a515-7b5abde239da","Type":"ContainerStarted","Data":"377ea34d27f71af9ef752812d802f7ffa69bdb0f443a062b3354e65885941481"}
Sep 10 13:31:43.956476 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.956452    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bhpvl" event={"ID":"c1f29aa0-318e-4f86-8500-4e3b68b848f7","Type":"ContainerStarted","Data":"90feb67896759915e555b2a3cf2620983cdf955cb42bd9ae8c4632e9feed7cc2"}
Sep 10 13:31:43.958281 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.958251    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5wjmf" event={"ID":"b42d2757-0d53-4232-8a6a-2039f0b93e0f","Type":"ContainerStarted","Data":"d27b3a5e8e55f56f361ce7f58c871b05e91bfa9117ec9346cc83e6cce8287e53"}
Sep 10 13:31:43.960290 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.960267    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s" event={"ID":"4ac14fc2-1f56-4e75-ab8f-04e54df2848e","Type":"ContainerStarted","Data":"86b0dff9e889b8c8e2f9150bdf513f0b4521ab7bd404aab5cbacdc874edb1e4a"}
Sep 10 13:31:43.960395 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.960303    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s" event={"ID":"4ac14fc2-1f56-4e75-ab8f-04e54df2848e","Type":"ContainerStarted","Data":"794fc2a883b61b5b4e154e9d93b8b6ac80821c9d7312080709beaccf27944417"}
Sep 10 13:31:43.960395 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.960320    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s" event={"ID":"4ac14fc2-1f56-4e75-ab8f-04e54df2848e","Type":"ContainerStarted","Data":"1518f668d10cace8f1d3e35ca05693089e9b4955ef05f72b2c4aeb83e0d031a9"}
Sep 10 13:31:43.961565 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.961546    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7" event={"ID":"2b8fa133-e000-42cb-af84-2198d386693b","Type":"ContainerStarted","Data":"ce9d3d62627fd57cc48f5c4798702cb4b89b825a2aa3fb9ef98a7222599489c2"}
Sep 10 13:31:43.961707 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.961694    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:43.963322 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.963302    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dbz76" event={"ID":"94abed11-8633-43cc-98a0-19223098fa40","Type":"ContainerStarted","Data":"9d2ccc6eaca6f8f62bf11564c295d97bb1679b2817d86a2e6098ce465fecee50"}
Sep 10 13:31:43.963415 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.963344    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dbz76" event={"ID":"94abed11-8633-43cc-98a0-19223098fa40","Type":"ContainerStarted","Data":"15fa1dc921aeaa8e2bbccde4a3d5c387690b14d4edbd52dbcbd8f543d0de2dd9"}
Sep 10 13:31:43.963473 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.963450    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:43.963516 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.963509    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7"
Sep 10 13:31:43.974842 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.974773    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7695494676-5q6jt" podStartSLOduration=1.591533027 podStartE2EDuration="9.974760755s" totalImagesPullingTime="8.383227728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:43.973681852 +0000 UTC m=+62.880239881" watchObservedRunningTime="2026-09-10 13:31:43.974760755 +0000 UTC m=+62.881318776"
Sep 10 13:31:43.999991 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:43.999952    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9f96db6b-gqs5s" podStartSLOduration=1.615626869 podStartE2EDuration="9.999938662s" totalImagesPullingTime="8.384311793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:43.999558545 +0000 UTC m=+62.906116573" watchObservedRunningTime="2026-09-10 13:31:43.999938662 +0000 UTC m=+62.906496691"
Sep 10 13:31:44.025229 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:44.025106    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5wjmf" podStartSLOduration=1.890536029 podStartE2EDuration="10.025091148s" totalImagesPullingTime="8.134555119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:31:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:44.024772448 +0000 UTC m=+62.931330477" watchObservedRunningTime="2026-09-10 13:31:44.025091148 +0000 UTC m=+62.931649178"
Sep 10 13:31:44.052078 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:44.052024    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d87c76d54-vw5q7" podStartSLOduration=1.655664393 podStartE2EDuration="10.052006484s" totalImagesPullingTime="8.396342091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:44.050817687 +0000 UTC m=+62.957375715" watchObservedRunningTime="2026-09-10 13:31:44.052006484 +0000 UTC m=+62.958564526"
Sep 10 13:31:44.119663 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:44.119448    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dbz76" podStartSLOduration=2.551832316 podStartE2EDuration="10.119430966s" totalImagesPullingTime="7.56759865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:44.119251471 +0000 UTC m=+63.025809521" watchObservedRunningTime="2026-09-10 13:31:44.119430966 +0000 UTC m=+63.025988994"
Sep 10 13:31:44.119663 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:44.119605    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bhpvl" podStartSLOduration=1.8679149769999999 podStartE2EDuration="10.119595641s" totalImagesPullingTime="8.251680664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:44.083803911 +0000 UTC m=+62.990361940" watchObservedRunningTime="2026-09-10 13:31:44.119595641 +0000 UTC m=+63.026153672"
Sep 10 13:31:46.353413 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.353352    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:46.353413 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.353417    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:46.356329 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.356311    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 13:31:46.357696 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.357676    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 13:31:46.366107 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.366084    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffad911d-9cef-4136-aeb5-ed9c6352add7-metrics-certs\") pod \"network-metrics-daemon-8qf55\" (UID: \"ffad911d-9cef-4136-aeb5-ed9c6352add7\") " pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:46.366107 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.366101    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e-original-pull-secret\") pod \"global-pull-secret-syncer-7jhj2\" (UID: \"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e\") " pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:46.453937 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.453886    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnl56\" (UniqueName: \"kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56\") pod \"network-check-target-tqlkj\" (UID: \"88d95652-29a9-4d30-b0e0-4795dc5e5077\") " pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:46.456548 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.456529    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 13:31:46.466989 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.466969    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 13:31:46.477103 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.477082    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnl56\" (UniqueName: \"kubernetes.io/projected/88d95652-29a9-4d30-b0e0-4795dc5e5077-kube-api-access-qnl56\") pod \"network-check-target-tqlkj\" (UID: \"88d95652-29a9-4d30-b0e0-4795dc5e5077\") " pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:46.583705 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.583675    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nc4jc\""
Sep 10 13:31:46.590009 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.589990    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-twwrf\""
Sep 10 13:31:46.591740 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.591726    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qf55"
Sep 10 13:31:46.592925 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.592910    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jhj2"
Sep 10 13:31:46.598133 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.598117    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:46.882038 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.881948    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7jhj2"]
Sep 10 13:31:46.883113 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.883091    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8qf55"]
Sep 10 13:31:46.886230 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:46.886207    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffad911d_9cef_4136_aeb5_ed9c6352add7.slice/crio-c766715f679d216306868c142e9b124fbf1b00870855b9c4f2862a702e3c91fa WatchSource:0}: Error finding container c766715f679d216306868c142e9b124fbf1b00870855b9c4f2862a702e3c91fa: Status 404 returned error can't find the container with id c766715f679d216306868c142e9b124fbf1b00870855b9c4f2862a702e3c91fa
Sep 10 13:31:46.886782 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:46.886761    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa5af7a5_42e5_4712_8eb8_cb5385bc6d6e.slice/crio-0a4e3f4474ffadf4ab3379e014c9a6e9c729211b3ad752e38391882b72bd895e WatchSource:0}: Error finding container 0a4e3f4474ffadf4ab3379e014c9a6e9c729211b3ad752e38391882b72bd895e: Status 404 returned error can't find the container with id 0a4e3f4474ffadf4ab3379e014c9a6e9c729211b3ad752e38391882b72bd895e
Sep 10 13:31:46.892334 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.892298    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tqlkj"]
Sep 10 13:31:46.894377 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:46.894357    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88d95652_29a9_4d30_b0e0_4795dc5e5077.slice/crio-b40bfe4e12e5c1fd7c7ba31f9d5bba3eb69248dd33bcdd4be47f1056a062909d WatchSource:0}: Error finding container b40bfe4e12e5c1fd7c7ba31f9d5bba3eb69248dd33bcdd4be47f1056a062909d: Status 404 returned error can't find the container with id b40bfe4e12e5c1fd7c7ba31f9d5bba3eb69248dd33bcdd4be47f1056a062909d
Sep 10 13:31:46.973600 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.973564    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7jhj2" event={"ID":"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e","Type":"ContainerStarted","Data":"0a4e3f4474ffadf4ab3379e014c9a6e9c729211b3ad752e38391882b72bd895e"}
Sep 10 13:31:46.974660 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.974634    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tqlkj" event={"ID":"88d95652-29a9-4d30-b0e0-4795dc5e5077","Type":"ContainerStarted","Data":"b40bfe4e12e5c1fd7c7ba31f9d5bba3eb69248dd33bcdd4be47f1056a062909d"}
Sep 10 13:31:46.975673 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:46.975656    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8qf55" event={"ID":"ffad911d-9cef-4136-aeb5-ed9c6352add7","Type":"ContainerStarted","Data":"c766715f679d216306868c142e9b124fbf1b00870855b9c4f2862a702e3c91fa"}
Sep 10 13:31:48.660469 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.660432    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h5v99"]
Sep 10 13:31:48.662943 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.662810    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.665957 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.665934    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 13:31:48.667098 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.667076    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 13:31:48.667512 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.667349    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 13:31:48.667512 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.667361    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tt9jm\""
Sep 10 13:31:48.669380 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.668854    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 13:31:48.674084 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.674010    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 13:31:48.675368 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.675328    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 13:31:48.771831 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.771790    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-tls\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.771831 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.771831    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-wtmp\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.772056 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.771888    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4tw4\" (UniqueName: \"kubernetes.io/projected/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-kube-api-access-v4tw4\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.772056 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.771943    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-sys\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.772056 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.771968    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.772056 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.771987    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-textfile\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.772056 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.772005    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-metrics-client-ca\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.772056 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.772041    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.772246 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.772143    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-root\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873128 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.872855    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873128 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.872927    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-textfile\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873128 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.872954    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-metrics-client-ca\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873128 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.872969    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873128 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.872998    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-root\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873128 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.873035    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-tls\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873535 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.873185    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-root\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873535 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.873187    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-wtmp\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873535 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.873258    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4tw4\" (UniqueName: \"kubernetes.io/projected/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-kube-api-access-v4tw4\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873535 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.873297    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-sys\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873535 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.873301    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-wtmp\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873535 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.873338    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-textfile\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.873535 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.873357    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-sys\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.874081 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.874050    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.874210 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.874052    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-metrics-client-ca\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.876031 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.875991    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.877047 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.877005    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-node-exporter-tls\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.890479 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.890429    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4tw4\" (UniqueName: \"kubernetes.io/projected/2e581b5d-2326-4c11-8b0b-2d79d91a28c0-kube-api-access-v4tw4\") pod \"node-exporter-h5v99\" (UID: \"2e581b5d-2326-4c11-8b0b-2d79d91a28c0\") " pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.974946 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.974846    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h5v99"
Sep 10 13:31:48.988899 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:31:48.988850    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e581b5d_2326_4c11_8b0b_2d79d91a28c0.slice/crio-6d8c4be85df1b0e58ca16cd6b91d5d249b19f4c09be9499a58a8ad494a42b32e WatchSource:0}: Error finding container 6d8c4be85df1b0e58ca16cd6b91d5d249b19f4c09be9499a58a8ad494a42b32e: Status 404 returned error can't find the container with id 6d8c4be85df1b0e58ca16cd6b91d5d249b19f4c09be9499a58a8ad494a42b32e
Sep 10 13:31:48.990533 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.990390    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8qf55" event={"ID":"ffad911d-9cef-4136-aeb5-ed9c6352add7","Type":"ContainerStarted","Data":"5ae801bbebea5f06a2fbb0bfd32e0c45a7e4adf3372c3513064b1cd4c0f42326"}
Sep 10 13:31:48.990533 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:48.990447    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8qf55" event={"ID":"ffad911d-9cef-4136-aeb5-ed9c6352add7","Type":"ContainerStarted","Data":"fb145f8befaa328cf56c0927f54d923ccb57a064e084a0b2bb372580ca2c646c"}
Sep 10 13:31:49.019954 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:49.018896    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8qf55" podStartSLOduration=65.957252948 podStartE2EDuration="1m7.018855495s" totalImagesPullingTime="1.061602547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:49.017944962 +0000 UTC m=+67.924502993" watchObservedRunningTime="2026-09-10 13:31:49.018855495 +0000 UTC m=+67.925413526"
Sep 10 13:31:49.996290 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:49.996249    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h5v99" event={"ID":"2e581b5d-2326-4c11-8b0b-2d79d91a28c0","Type":"ContainerStarted","Data":"6d8c4be85df1b0e58ca16cd6b91d5d249b19f4c09be9499a58a8ad494a42b32e"}
Sep 10 13:31:51.002049 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:51.002013    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tqlkj" event={"ID":"88d95652-29a9-4d30-b0e0-4795dc5e5077","Type":"ContainerStarted","Data":"d42f2e2b7fcf9c8cafaacb2516b96c48575535d99281748fdea95941e7866a44"}
Sep 10 13:31:51.002439 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:51.002141    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:31:52.008048 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:52.008007    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7jhj2" event={"ID":"fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e","Type":"ContainerStarted","Data":"ca53e7f7ccb843465a4dd515bc5e3ae2e61285e11965a9eceb1c1686d877b8ad"}
Sep 10 13:31:52.045812 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:52.045753    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7jhj2" podStartSLOduration=65.384101545 podStartE2EDuration="1m10.045733702s" totalImagesPullingTime="4.661632157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:52.04440865 +0000 UTC m=+70.950966679" watchObservedRunningTime="2026-09-10 13:31:52.045733702 +0000 UTC m=+70.952291735"
Sep 10 13:31:52.046331 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:52.046304    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tqlkj" podStartSLOduration=66.802979793 podStartE2EDuration="1m10.046295159s" totalImagesPullingTime="3.243315366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:51.037267757 +0000 UTC m=+69.943825787" watchObservedRunningTime="2026-09-10 13:31:52.046295159 +0000 UTC m=+70.952853188"
Sep 10 13:31:53.013420 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:53.013383    2549 generic.go:356] "Generic (PLEG): container finished" podID="2e581b5d-2326-4c11-8b0b-2d79d91a28c0" containerID="25f7fe4f904015253ed206c1183c732ee074b7c2a0fcc593fdc81535bba0d459" exitCode=0
Sep 10 13:31:53.013988 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:53.013472    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h5v99" event={"ID":"2e581b5d-2326-4c11-8b0b-2d79d91a28c0","Type":"ContainerDied","Data":"25f7fe4f904015253ed206c1183c732ee074b7c2a0fcc593fdc81535bba0d459"}
Sep 10 13:31:53.969119 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:53.969089    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dbz76"
Sep 10 13:31:54.018857 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:54.018820    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h5v99" event={"ID":"2e581b5d-2326-4c11-8b0b-2d79d91a28c0","Type":"ContainerStarted","Data":"80a6ca302522f76859bc8b80ba0b75544aa1b39b58c5dc83006eb87de45bac85"}
Sep 10 13:31:54.019239 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:54.018893    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h5v99" event={"ID":"2e581b5d-2326-4c11-8b0b-2d79d91a28c0","Type":"ContainerStarted","Data":"7c807e7d91f0952b129481179ceefb695c583fd2dd887449648fbe4cf0c9de77"}
Sep 10 13:31:54.074070 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:54.074019    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h5v99" podStartSLOduration=2.808964938 podStartE2EDuration="6.074003326s" totalImagesPullingTime="3.207162139s" totalInitContainerRuntime="57.876249ms" isStatefulPod=true podCreationTimestamp="2026-09-10 13:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:54.073936526 +0000 UTC m=+72.980494585" watchObservedRunningTime="2026-09-10 13:31:54.074003326 +0000 UTC m=+72.980561374"
Sep 10 13:31:57.938683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:31:57.938655    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c947ff77-zhw2x"
Sep 10 13:32:22.010694 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:32:22.010655    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tqlkj"
Sep 10 13:35:41.480671 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:35:41.480641    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:35:41.481788 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:35:41.481766    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:35:41.483833 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:35:41.483810    2549 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 13:36:08.773825 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.773791    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"]
Sep 10 13:36:08.779114 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.779091    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.783220 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.783199    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 13:36:08.783346 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.783319    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 13:36:08.783589 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.783569    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 13:36:08.783679 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.783592    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sf2hc\""
Sep 10 13:36:08.784417 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.784398    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 13:36:08.799852 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.799833    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"]
Sep 10 13:36:08.826792 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.826764    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae74f9c9-a360-443b-b9c1-07457225c146-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.826966 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.826847    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae74f9c9-a360-443b-b9c1-07457225c146-webhook-cert\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.826966 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.826903    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6h5r\" (UniqueName: \"kubernetes.io/projected/ae74f9c9-a360-443b-b9c1-07457225c146-kube-api-access-d6h5r\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.826966 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.826948    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae74f9c9-a360-443b-b9c1-07457225c146-tmp\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.827115 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.827018    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae74f9c9-a360-443b-b9c1-07457225c146-manifests\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.928340 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.928306    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6h5r\" (UniqueName: \"kubernetes.io/projected/ae74f9c9-a360-443b-b9c1-07457225c146-kube-api-access-d6h5r\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.928518 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.928354    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae74f9c9-a360-443b-b9c1-07457225c146-tmp\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.928518 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.928384    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae74f9c9-a360-443b-b9c1-07457225c146-manifests\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.928518 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.928410    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae74f9c9-a360-443b-b9c1-07457225c146-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.928518 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.928432    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae74f9c9-a360-443b-b9c1-07457225c146-webhook-cert\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.928827 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.928790    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae74f9c9-a360-443b-b9c1-07457225c146-manifests\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.930748 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.930717    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae74f9c9-a360-443b-b9c1-07457225c146-tmp\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.930860 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.930812    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae74f9c9-a360-443b-b9c1-07457225c146-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.931007 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.930989    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae74f9c9-a360-443b-b9c1-07457225c146-webhook-cert\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:08.937694 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:08.937674    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6h5r\" (UniqueName: \"kubernetes.io/projected/ae74f9c9-a360-443b-b9c1-07457225c146-kube-api-access-d6h5r\") pod \"opendatahub-operator-controller-manager-94d598c87-fdn5q\" (UID: \"ae74f9c9-a360-443b-b9c1-07457225c146\") " pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:09.089229 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:09.089136    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:09.387371 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:09.387322    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"]
Sep 10 13:36:09.389558 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:36:09.389527    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae74f9c9_a360_443b_b9c1_07457225c146.slice/crio-2916f4396ce59828784b2dc0185be27a5935220533e4c6a6dfed31b351c92d04 WatchSource:0}: Error finding container 2916f4396ce59828784b2dc0185be27a5935220533e4c6a6dfed31b351c92d04: Status 404 returned error can't find the container with id 2916f4396ce59828784b2dc0185be27a5935220533e4c6a6dfed31b351c92d04
Sep 10 13:36:09.391231 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:09.391212    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:36:09.825628 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:09.825593    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q" event={"ID":"ae74f9c9-a360-443b-b9c1-07457225c146","Type":"ContainerStarted","Data":"2916f4396ce59828784b2dc0185be27a5935220533e4c6a6dfed31b351c92d04"}
Sep 10 13:36:12.838618 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:12.838581    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q" event={"ID":"ae74f9c9-a360-443b-b9c1-07457225c146","Type":"ContainerStarted","Data":"7915a0ff84c391a9da7464fe29e5f69cb11933cb44839494c763526cd8a594b1"}
Sep 10 13:36:13.843639 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:13.843546    2549 generic.go:356] "Generic (PLEG): container finished" podID="ae74f9c9-a360-443b-b9c1-07457225c146" containerID="7915a0ff84c391a9da7464fe29e5f69cb11933cb44839494c763526cd8a594b1" exitCode=0
Sep 10 13:36:13.843639 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:13.843589    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q" event={"ID":"ae74f9c9-a360-443b-b9c1-07457225c146","Type":"ContainerDied","Data":"7915a0ff84c391a9da7464fe29e5f69cb11933cb44839494c763526cd8a594b1"}
Sep 10 13:36:14.849091 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:14.849058    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q" event={"ID":"ae74f9c9-a360-443b-b9c1-07457225c146","Type":"ContainerStarted","Data":"24ede83266dc77b3f7295555ba8ec43ed045240e2e10d3b615b493140372c88e"}
Sep 10 13:36:14.849445 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:14.849177    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:14.879843 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:14.879799    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q" podStartSLOduration=2.007524673 podStartE2EDuration="6.879783839s" totalImagesPullingTime="3.850889874s" totalInitContainerRuntime="1.021369292s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:14.878289998 +0000 UTC m=+333.784848025" watchObservedRunningTime="2026-09-10 13:36:14.879783839 +0000 UTC m=+333.786341869"
Sep 10 13:36:25.854171 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:25.854141    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-94d598c87-fdn5q"
Sep 10 13:36:32.527249 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.527210    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"]
Sep 10 13:36:32.536754 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.536730    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:32.539525 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.539500    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 13:36:32.539525 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.539500    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 13:36:32.540008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.539988    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 13:36:32.545802 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.545781    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 13:36:32.548289 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.548265    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"]
Sep 10 13:36:32.604544 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.604515    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/735169af-6ec4-421c-a26a-22c3532b15ff-tmp\") pod \"kube-auth-proxy-66cb58b9b9-2kghf\" (UID: \"735169af-6ec4-421c-a26a-22c3532b15ff\") " pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:32.604687 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.604570    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/735169af-6ec4-421c-a26a-22c3532b15ff-tls-certs\") pod \"kube-auth-proxy-66cb58b9b9-2kghf\" (UID: \"735169af-6ec4-421c-a26a-22c3532b15ff\") " pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:32.604687 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.604587    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5pks\" (UniqueName: \"kubernetes.io/projected/735169af-6ec4-421c-a26a-22c3532b15ff-kube-api-access-w5pks\") pod \"kube-auth-proxy-66cb58b9b9-2kghf\" (UID: \"735169af-6ec4-421c-a26a-22c3532b15ff\") " pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:32.705498 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.705463    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/735169af-6ec4-421c-a26a-22c3532b15ff-tls-certs\") pod \"kube-auth-proxy-66cb58b9b9-2kghf\" (UID: \"735169af-6ec4-421c-a26a-22c3532b15ff\") " pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:32.705498 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.705498    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5pks\" (UniqueName: \"kubernetes.io/projected/735169af-6ec4-421c-a26a-22c3532b15ff-kube-api-access-w5pks\") pod \"kube-auth-proxy-66cb58b9b9-2kghf\" (UID: \"735169af-6ec4-421c-a26a-22c3532b15ff\") " pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:32.705700 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.705541    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/735169af-6ec4-421c-a26a-22c3532b15ff-tmp\") pod \"kube-auth-proxy-66cb58b9b9-2kghf\" (UID: \"735169af-6ec4-421c-a26a-22c3532b15ff\") " pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:32.707789 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.707764    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/735169af-6ec4-421c-a26a-22c3532b15ff-tmp\") pod \"kube-auth-proxy-66cb58b9b9-2kghf\" (UID: \"735169af-6ec4-421c-a26a-22c3532b15ff\") " pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:32.708021 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.708002    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/735169af-6ec4-421c-a26a-22c3532b15ff-tls-certs\") pod \"kube-auth-proxy-66cb58b9b9-2kghf\" (UID: \"735169af-6ec4-421c-a26a-22c3532b15ff\") " pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:32.718809 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.718783    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5pks\" (UniqueName: \"kubernetes.io/projected/735169af-6ec4-421c-a26a-22c3532b15ff-kube-api-access-w5pks\") pod \"kube-auth-proxy-66cb58b9b9-2kghf\" (UID: \"735169af-6ec4-421c-a26a-22c3532b15ff\") " pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:32.848170 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:32.848080    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"
Sep 10 13:36:33.125097 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:33.125007    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf"]
Sep 10 13:36:33.128765 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:36:33.128742    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735169af_6ec4_421c_a26a_22c3532b15ff.slice/crio-3e78285913da7b0a53d5e47fa66462b581da21df8875081df47aaf68beebaa7d WatchSource:0}: Error finding container 3e78285913da7b0a53d5e47fa66462b581da21df8875081df47aaf68beebaa7d: Status 404 returned error can't find the container with id 3e78285913da7b0a53d5e47fa66462b581da21df8875081df47aaf68beebaa7d
Sep 10 13:36:33.910114 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:33.910031    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf" event={"ID":"735169af-6ec4-421c-a26a-22c3532b15ff","Type":"ContainerStarted","Data":"3e78285913da7b0a53d5e47fa66462b581da21df8875081df47aaf68beebaa7d"}
Sep 10 13:36:34.191385 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.191295    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"]
Sep 10 13:36:34.193846 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.193822    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.196540 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.196364    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 13:36:34.196663 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.196625    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 13:36:34.196728 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.196683    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 13:36:34.197107 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.196923    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 13:36:34.197107 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.196942    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-84mmp\""
Sep 10 13:36:34.213775 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.213726    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"]
Sep 10 13:36:34.319140 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.319109    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbnbn\" (UniqueName: \"kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-kube-api-access-zbnbn\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.319317 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.319162    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.319317 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.319187    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.319317 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.319278    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.319317 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.319311    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.319685 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.319364    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.319685 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.319395    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9783984e-aff2-4cc8-accb-546e01076027-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.420638 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.420607    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.420638 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.420641    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9783984e-aff2-4cc8-accb-546e01076027-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.420890 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.420782    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnbn\" (UniqueName: \"kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-kube-api-access-zbnbn\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.420890 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.420832    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.420890 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.420876    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.421038 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.420941    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.421038 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.420965    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.421577 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.421548    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.423476 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.423451    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.423476 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.423468    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9783984e-aff2-4cc8-accb-546e01076027-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.423651 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.423563    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.423721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.423701    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.430571 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.430538    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.431011 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.430988    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbnbn\" (UniqueName: \"kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-kube-api-access-zbnbn\") pod \"istiod-openshift-gateway-7b7995d5cd-sjdj9\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.507915 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.507818    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:34.814301 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:34.814193    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"]
Sep 10 13:36:36.042974 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:36:36.042942    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9783984e_aff2_4cc8_accb_546e01076027.slice/crio-f368902e2bd14af7e5d5358c10f85b360d524260ba01bd77564e47059630e07a WatchSource:0}: Error finding container f368902e2bd14af7e5d5358c10f85b360d524260ba01bd77564e47059630e07a: Status 404 returned error can't find the container with id f368902e2bd14af7e5d5358c10f85b360d524260ba01bd77564e47059630e07a
Sep 10 13:36:36.925465 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:36.925099    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf" event={"ID":"735169af-6ec4-421c-a26a-22c3532b15ff","Type":"ContainerStarted","Data":"6a0c6acdfe65860cd4861bbfddf94b8c7447686e6c8197eed7968a40ef7f88c9"}
Sep 10 13:36:36.926539 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:36.926508    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9" event={"ID":"9783984e-aff2-4cc8-accb-546e01076027","Type":"ContainerStarted","Data":"f368902e2bd14af7e5d5358c10f85b360d524260ba01bd77564e47059630e07a"}
Sep 10 13:36:36.958691 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:36.958608    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66cb58b9b9-2kghf" podStartSLOduration=2.014632515 podStartE2EDuration="4.958586086s" totalImagesPullingTime="2.943953571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:36.958051073 +0000 UTC m=+355.864609104" watchObservedRunningTime="2026-09-10 13:36:36.958586086 +0000 UTC m=+355.865144098"
Sep 10 13:36:38.461579 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:38.461537    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 13:36:38.461824 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:38.461606    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 13:36:38.935438 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:38.935403    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9" event={"ID":"9783984e-aff2-4cc8-accb-546e01076027","Type":"ContainerStarted","Data":"50b85f33ae7eafeadbc10725bf2c8dc79619705ddc3c2c30602ec179ac026879"}
Sep 10 13:36:38.935639 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:38.935497    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:38.959377 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:38.959332    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9" podStartSLOduration=2.542952552 podStartE2EDuration="4.959316598s" totalImagesPullingTime="2.416364046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:38.957405881 +0000 UTC m=+357.863963934" watchObservedRunningTime="2026-09-10 13:36:38.959316598 +0000 UTC m=+357.865874628"
Sep 10 13:36:39.941005 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:39.940972    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:48.290014 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.289982    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"]
Sep 10 13:36:48.293248 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.293226    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.297603 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.297578    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 13:36:48.298075 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.298056    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 13:36:48.298403 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.298387    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 13:36:48.298855 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.298836    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 13:36:48.298970 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.298894    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 13:36:48.298970 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.298937    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nlz28\""
Sep 10 13:36:48.310565 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.310540    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"]
Sep 10 13:36:48.445365 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.445328    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/66b89271-8918-44c5-b25b-630780289f26-manager-config\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.445532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.445373    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prnxs\" (UniqueName: \"kubernetes.io/projected/66b89271-8918-44c5-b25b-630780289f26-kube-api-access-prnxs\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.445532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.445487    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66b89271-8918-44c5-b25b-630780289f26-cert\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.445616 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.445531    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/66b89271-8918-44c5-b25b-630780289f26-metrics-cert\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.546987 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.546906    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66b89271-8918-44c5-b25b-630780289f26-cert\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.546987 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.546954    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/66b89271-8918-44c5-b25b-630780289f26-metrics-cert\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.547163 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.546997    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/66b89271-8918-44c5-b25b-630780289f26-manager-config\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.547163 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.547030    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prnxs\" (UniqueName: \"kubernetes.io/projected/66b89271-8918-44c5-b25b-630780289f26-kube-api-access-prnxs\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.547678 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.547654    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/66b89271-8918-44c5-b25b-630780289f26-manager-config\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.549449 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.549424    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/66b89271-8918-44c5-b25b-630780289f26-metrics-cert\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.549541 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.549465    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66b89271-8918-44c5-b25b-630780289f26-cert\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.555901 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.555860    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prnxs\" (UniqueName: \"kubernetes.io/projected/66b89271-8918-44c5-b25b-630780289f26-kube-api-access-prnxs\") pod \"lws-controller-manager-59689cfc58-fvqwx\" (UID: \"66b89271-8918-44c5-b25b-630780289f26\") " pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.601922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.601897    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:48.873541 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.873460    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"]
Sep 10 13:36:48.876121 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:36:48.876095    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b89271_8918_44c5_b25b_630780289f26.slice/crio-109ea62627e10e639e1cc7dc3419f84c01c2978d696a7953f5da9ab6e1bf94af WatchSource:0}: Error finding container 109ea62627e10e639e1cc7dc3419f84c01c2978d696a7953f5da9ab6e1bf94af: Status 404 returned error can't find the container with id 109ea62627e10e639e1cc7dc3419f84c01c2978d696a7953f5da9ab6e1bf94af
Sep 10 13:36:48.976360 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:48.976321    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx" event={"ID":"66b89271-8918-44c5-b25b-630780289f26","Type":"ContainerStarted","Data":"109ea62627e10e639e1cc7dc3419f84c01c2978d696a7953f5da9ab6e1bf94af"}
Sep 10 13:36:51.989199 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:51.989159    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx" event={"ID":"66b89271-8918-44c5-b25b-630780289f26","Type":"ContainerStarted","Data":"0e96caf6ddeb817564f67f1bbabe702134a8c093955d29b75a98798d70e9728c"}
Sep 10 13:36:51.989627 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:51.989233    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:36:52.025711 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:52.025662    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx" podStartSLOduration=1.7111092060000002 podStartE2EDuration="4.025645324s" totalImagesPullingTime="2.314536118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:52.024935306 +0000 UTC m=+370.931493359" watchObservedRunningTime="2026-09-10 13:36:52.025645324 +0000 UTC m=+370.932203353"
Sep 10 13:36:52.867446 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:52.867405    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"]
Sep 10 13:36:52.867676 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:52.867649    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9" podUID="9783984e-aff2-4cc8-accb-546e01076027" containerName="discovery" containerID="cri-o://50b85f33ae7eafeadbc10725bf2c8dc79619705ddc3c2c30602ec179ac026879" gracePeriod=30
Sep 10 13:36:52.995040 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:52.995009    2549 generic.go:356] "Generic (PLEG): container finished" podID="9783984e-aff2-4cc8-accb-546e01076027" containerID="50b85f33ae7eafeadbc10725bf2c8dc79619705ddc3c2c30602ec179ac026879" exitCode=0
Sep 10 13:36:52.995395 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:52.995097    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9" event={"ID":"9783984e-aff2-4cc8-accb-546e01076027","Type":"ContainerDied","Data":"50b85f33ae7eafeadbc10725bf2c8dc79619705ddc3c2c30602ec179ac026879"}
Sep 10 13:36:53.103604 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.103580    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:53.286383 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.286350    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-kubeconfig\") pod \"9783984e-aff2-4cc8-accb-546e01076027\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") "
Sep 10 13:36:53.286383 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.286391    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9783984e-aff2-4cc8-accb-546e01076027-local-certs\") pod \"9783984e-aff2-4cc8-accb-546e01076027\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") "
Sep 10 13:36:53.286650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.286422    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-istio-token\") pod \"9783984e-aff2-4cc8-accb-546e01076027\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") "
Sep 10 13:36:53.286650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.286439    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-cacerts\") pod \"9783984e-aff2-4cc8-accb-546e01076027\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") "
Sep 10 13:36:53.286650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.286472    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-ca-configmap\") pod \"9783984e-aff2-4cc8-accb-546e01076027\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") "
Sep 10 13:36:53.286650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.286502    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-dns-cert\") pod \"9783984e-aff2-4cc8-accb-546e01076027\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") "
Sep 10 13:36:53.286650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.286526    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbnbn\" (UniqueName: \"kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-kube-api-access-zbnbn\") pod \"9783984e-aff2-4cc8-accb-546e01076027\" (UID: \"9783984e-aff2-4cc8-accb-546e01076027\") "
Sep 10 13:36:53.287029 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.286969    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "9783984e-aff2-4cc8-accb-546e01076027" (UID: "9783984e-aff2-4cc8-accb-546e01076027"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:36:53.288839 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.288793    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9783984e-aff2-4cc8-accb-546e01076027-local-certs" (OuterVolumeSpecName: "local-certs") pod "9783984e-aff2-4cc8-accb-546e01076027" (UID: "9783984e-aff2-4cc8-accb-546e01076027"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:36:53.288973 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.288900    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-kube-api-access-zbnbn" (OuterVolumeSpecName: "kube-api-access-zbnbn") pod "9783984e-aff2-4cc8-accb-546e01076027" (UID: "9783984e-aff2-4cc8-accb-546e01076027"). InnerVolumeSpecName "kube-api-access-zbnbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:36:53.289130 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.289106    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-cacerts" (OuterVolumeSpecName: "cacerts") pod "9783984e-aff2-4cc8-accb-546e01076027" (UID: "9783984e-aff2-4cc8-accb-546e01076027"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:36:53.289219 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.289129    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "9783984e-aff2-4cc8-accb-546e01076027" (UID: "9783984e-aff2-4cc8-accb-546e01076027"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:36:53.289219 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.289142    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-istio-token" (OuterVolumeSpecName: "istio-token") pod "9783984e-aff2-4cc8-accb-546e01076027" (UID: "9783984e-aff2-4cc8-accb-546e01076027"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:36:53.289219 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.289174    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "9783984e-aff2-4cc8-accb-546e01076027" (UID: "9783984e-aff2-4cc8-accb-546e01076027"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:36:53.387273 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.387218    2549 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-kubeconfig\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:36:53.387273 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.387264    2549 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9783984e-aff2-4cc8-accb-546e01076027-local-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:36:53.387273 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.387275    2549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:36:53.387273 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.387282    2549 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-cacerts\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:36:53.387273 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.387291    2549 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-ca-configmap\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:36:53.387570 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.387300    2549 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9783984e-aff2-4cc8-accb-546e01076027-istio-csr-dns-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:36:53.387570 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:53.387308    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbnbn\" (UniqueName: \"kubernetes.io/projected/9783984e-aff2-4cc8-accb-546e01076027-kube-api-access-zbnbn\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:36:54.000104 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:54.000003    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9" event={"ID":"9783984e-aff2-4cc8-accb-546e01076027","Type":"ContainerDied","Data":"f368902e2bd14af7e5d5358c10f85b360d524260ba01bd77564e47059630e07a"}
Sep 10 13:36:54.000104 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:54.000052    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"
Sep 10 13:36:54.000104 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:54.000074    2549 scope.go:117] "RemoveContainer" containerID="50b85f33ae7eafeadbc10725bf2c8dc79619705ddc3c2c30602ec179ac026879"
Sep 10 13:36:54.023892 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:54.023846    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"]
Sep 10 13:36:54.030783 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:54.030753    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sjdj9"]
Sep 10 13:36:55.672765 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:36:55.672728    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9783984e-aff2-4cc8-accb-546e01076027" path="/var/lib/kubelet/pods/9783984e-aff2-4cc8-accb-546e01076027/volumes"
Sep 10 13:37:02.997661 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:37:02.997631    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59689cfc58-fvqwx"
Sep 10 13:38:12.799143 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.799009    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"]
Sep 10 13:38:12.799601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.799362    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9783984e-aff2-4cc8-accb-546e01076027" containerName="discovery"
Sep 10 13:38:12.799601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.799377    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="9783984e-aff2-4cc8-accb-546e01076027" containerName="discovery"
Sep 10 13:38:12.799601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.799421    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="9783984e-aff2-4cc8-accb-546e01076027" containerName="discovery"
Sep 10 13:38:12.802129 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.802108    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:12.806242 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.806220    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 13:38:12.806414 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.806396    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 13:38:12.807337 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.807319    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 13:38:12.807572 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.807557    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ph8jr\""
Sep 10 13:38:12.807662 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.807612    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 13:38:12.817071 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.817049    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"]
Sep 10 13:38:12.978623 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.978587    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt5g7\" (UniqueName: \"kubernetes.io/projected/5d810f3c-dfc9-431d-9d25-963f8eedb69f-kube-api-access-qt5g7\") pod \"kuadrant-console-plugin-fbdd4f445-9wg8q\" (UID: \"5d810f3c-dfc9-431d-9d25-963f8eedb69f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:12.978802 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.978650    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d810f3c-dfc9-431d-9d25-963f8eedb69f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9wg8q\" (UID: \"5d810f3c-dfc9-431d-9d25-963f8eedb69f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:12.978802 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:12.978672    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5d810f3c-dfc9-431d-9d25-963f8eedb69f-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9wg8q\" (UID: \"5d810f3c-dfc9-431d-9d25-963f8eedb69f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:13.079920 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:13.079819    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d810f3c-dfc9-431d-9d25-963f8eedb69f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9wg8q\" (UID: \"5d810f3c-dfc9-431d-9d25-963f8eedb69f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:13.079920 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:13.079856    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5d810f3c-dfc9-431d-9d25-963f8eedb69f-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9wg8q\" (UID: \"5d810f3c-dfc9-431d-9d25-963f8eedb69f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:13.080096 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:13.079929    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt5g7\" (UniqueName: \"kubernetes.io/projected/5d810f3c-dfc9-431d-9d25-963f8eedb69f-kube-api-access-qt5g7\") pod \"kuadrant-console-plugin-fbdd4f445-9wg8q\" (UID: \"5d810f3c-dfc9-431d-9d25-963f8eedb69f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:13.080096 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:38:13.079974    2549 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 10 13:38:13.080096 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:38:13.080051    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d810f3c-dfc9-431d-9d25-963f8eedb69f-plugin-serving-cert podName:5d810f3c-dfc9-431d-9d25-963f8eedb69f nodeName:}" failed. No retries permitted until 2026-09-10 13:38:13.580034987 +0000 UTC m=+452.486592996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/5d810f3c-dfc9-431d-9d25-963f8eedb69f-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-9wg8q" (UID: "5d810f3c-dfc9-431d-9d25-963f8eedb69f") : secret "plugin-serving-cert" not found
Sep 10 13:38:13.080602 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:13.080583    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5d810f3c-dfc9-431d-9d25-963f8eedb69f-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9wg8q\" (UID: \"5d810f3c-dfc9-431d-9d25-963f8eedb69f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:13.113138 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:13.113109    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt5g7\" (UniqueName: \"kubernetes.io/projected/5d810f3c-dfc9-431d-9d25-963f8eedb69f-kube-api-access-qt5g7\") pod \"kuadrant-console-plugin-fbdd4f445-9wg8q\" (UID: \"5d810f3c-dfc9-431d-9d25-963f8eedb69f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:13.585081 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:13.585043    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d810f3c-dfc9-431d-9d25-963f8eedb69f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9wg8q\" (UID: \"5d810f3c-dfc9-431d-9d25-963f8eedb69f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:13.587535 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:13.587508    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d810f3c-dfc9-431d-9d25-963f8eedb69f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9wg8q\" (UID: \"5d810f3c-dfc9-431d-9d25-963f8eedb69f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:13.710831 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:13.710796    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"
Sep 10 13:38:13.990749 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:13.990714    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q"]
Sep 10 13:38:13.993704 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:38:13.993676    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d810f3c_dfc9_431d_9d25_963f8eedb69f.slice/crio-b2f5755ccbfde7663997fe62af70babdd29e66474101dca5549c3a5a915b6348 WatchSource:0}: Error finding container b2f5755ccbfde7663997fe62af70babdd29e66474101dca5549c3a5a915b6348: Status 404 returned error can't find the container with id b2f5755ccbfde7663997fe62af70babdd29e66474101dca5549c3a5a915b6348
Sep 10 13:38:14.286440 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:14.286357    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q" event={"ID":"5d810f3c-dfc9-431d-9d25-963f8eedb69f","Type":"ContainerStarted","Data":"b2f5755ccbfde7663997fe62af70babdd29e66474101dca5549c3a5a915b6348"}
Sep 10 13:38:19.309769 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:19.309742    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q" event={"ID":"5d810f3c-dfc9-431d-9d25-963f8eedb69f","Type":"ContainerStarted","Data":"084b61f5a8202a11012da94373bb4b610912d77c1a864d352aa46314dd1e14ba"}
Sep 10 13:38:19.333569 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:38:19.330409    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9wg8q" podStartSLOduration=3.067516241 podStartE2EDuration="7.330386824s" totalImagesPullingTime="4.262870583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:38:19.328098223 +0000 UTC m=+458.234656250" watchObservedRunningTime="2026-09-10 13:38:19.330386824 +0000 UTC m=+458.236944856"
Sep 10 13:39:08.178289 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.178256    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8hwlx"]
Sep 10 13:39:08.184767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.184733    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx"
Sep 10 13:39:08.187524 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.187494    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-j65wv\""
Sep 10 13:39:08.195108 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.195088    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8hwlx"]
Sep 10 13:39:08.271852 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.271822    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pjv6\" (UniqueName: \"kubernetes.io/projected/7eb16171-d8b0-458e-9ad3-68f85c0d25dc-kube-api-access-2pjv6\") pod \"authorino-7bcfd8f9cd-8hwlx\" (UID: \"7eb16171-d8b0-458e-9ad3-68f85c0d25dc\") " pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx"
Sep 10 13:39:08.361780 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.361744    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-fr28j"]
Sep 10 13:39:08.364650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.364634    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-fr28j"
Sep 10 13:39:08.372484 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.372465    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pjv6\" (UniqueName: \"kubernetes.io/projected/7eb16171-d8b0-458e-9ad3-68f85c0d25dc-kube-api-access-2pjv6\") pod \"authorino-7bcfd8f9cd-8hwlx\" (UID: \"7eb16171-d8b0-458e-9ad3-68f85c0d25dc\") " pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx"
Sep 10 13:39:08.384397 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.384369    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-fr28j"]
Sep 10 13:39:08.387547 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.387518    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pjv6\" (UniqueName: \"kubernetes.io/projected/7eb16171-d8b0-458e-9ad3-68f85c0d25dc-kube-api-access-2pjv6\") pod \"authorino-7bcfd8f9cd-8hwlx\" (UID: \"7eb16171-d8b0-458e-9ad3-68f85c0d25dc\") " pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx"
Sep 10 13:39:08.473055 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.472986    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6vjz\" (UniqueName: \"kubernetes.io/projected/d16cc6e9-aa23-458d-b29f-ffb01e0e6204-kube-api-access-b6vjz\") pod \"authorino-59c974bd48-fr28j\" (UID: \"d16cc6e9-aa23-458d-b29f-ffb01e0e6204\") " pod="kuadrant-system/authorino-59c974bd48-fr28j"
Sep 10 13:39:08.493214 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.493191    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx"
Sep 10 13:39:08.573544 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.573512    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6vjz\" (UniqueName: \"kubernetes.io/projected/d16cc6e9-aa23-458d-b29f-ffb01e0e6204-kube-api-access-b6vjz\") pod \"authorino-59c974bd48-fr28j\" (UID: \"d16cc6e9-aa23-458d-b29f-ffb01e0e6204\") " pod="kuadrant-system/authorino-59c974bd48-fr28j"
Sep 10 13:39:08.583796 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.583767    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6vjz\" (UniqueName: \"kubernetes.io/projected/d16cc6e9-aa23-458d-b29f-ffb01e0e6204-kube-api-access-b6vjz\") pod \"authorino-59c974bd48-fr28j\" (UID: \"d16cc6e9-aa23-458d-b29f-ffb01e0e6204\") " pod="kuadrant-system/authorino-59c974bd48-fr28j"
Sep 10 13:39:08.673129 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.673100    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-fr28j"
Sep 10 13:39:08.762435 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.762367    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8hwlx"]
Sep 10 13:39:08.765222 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:39:08.765197    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7eb16171_d8b0_458e_9ad3_68f85c0d25dc.slice/crio-1cfc4360cfa350f088b002dcf0c7d17b88084e4f31bceb9131878364eddf4bab WatchSource:0}: Error finding container 1cfc4360cfa350f088b002dcf0c7d17b88084e4f31bceb9131878364eddf4bab: Status 404 returned error can't find the container with id 1cfc4360cfa350f088b002dcf0c7d17b88084e4f31bceb9131878364eddf4bab
Sep 10 13:39:08.942773 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:08.942740    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-fr28j"]
Sep 10 13:39:08.946141 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:39:08.946114    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd16cc6e9_aa23_458d_b29f_ffb01e0e6204.slice/crio-a976474b93b5dfeb9346f9b84e5fb131ae637c0b8bea32f2e4d8a6c32177cce3 WatchSource:0}: Error finding container a976474b93b5dfeb9346f9b84e5fb131ae637c0b8bea32f2e4d8a6c32177cce3: Status 404 returned error can't find the container with id a976474b93b5dfeb9346f9b84e5fb131ae637c0b8bea32f2e4d8a6c32177cce3
Sep 10 13:39:09.483300 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:09.483266    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-fr28j" event={"ID":"d16cc6e9-aa23-458d-b29f-ffb01e0e6204","Type":"ContainerStarted","Data":"a976474b93b5dfeb9346f9b84e5fb131ae637c0b8bea32f2e4d8a6c32177cce3"}
Sep 10 13:39:09.484401 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:09.484378    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx" event={"ID":"7eb16171-d8b0-458e-9ad3-68f85c0d25dc","Type":"ContainerStarted","Data":"1cfc4360cfa350f088b002dcf0c7d17b88084e4f31bceb9131878364eddf4bab"}
Sep 10 13:39:12.498549 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:12.498502    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-fr28j" event={"ID":"d16cc6e9-aa23-458d-b29f-ffb01e0e6204","Type":"ContainerStarted","Data":"b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664"}
Sep 10 13:39:12.499991 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:12.499967    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx" event={"ID":"7eb16171-d8b0-458e-9ad3-68f85c0d25dc","Type":"ContainerStarted","Data":"1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f"}
Sep 10 13:39:12.519781 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:12.519734    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-fr28j" podStartSLOduration=1.43871728 podStartE2EDuration="4.519722118s" totalImagesPullingTime="3.081004838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:12.519305355 +0000 UTC m=+511.425863387" watchObservedRunningTime="2026-09-10 13:39:12.519722118 +0000 UTC m=+511.426280217"
Sep 10 13:39:12.537557 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:12.537493    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx" podStartSLOduration=1.275952629 podStartE2EDuration="4.537472897s" totalImagesPullingTime="3.261520268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:12.537301647 +0000 UTC m=+511.443859679" watchObservedRunningTime="2026-09-10 13:39:12.537472897 +0000 UTC m=+511.444030931"
Sep 10 13:39:12.571460 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:12.571428    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8hwlx"]
Sep 10 13:39:14.508169 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:14.508130    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx" podUID="7eb16171-d8b0-458e-9ad3-68f85c0d25dc" containerName="authorino" containerID="cri-o://1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f" gracePeriod=30
Sep 10 13:39:14.757141 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:14.757112    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx"
Sep 10 13:39:14.822420 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:14.822320    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pjv6\" (UniqueName: \"kubernetes.io/projected/7eb16171-d8b0-458e-9ad3-68f85c0d25dc-kube-api-access-2pjv6\") pod \"7eb16171-d8b0-458e-9ad3-68f85c0d25dc\" (UID: \"7eb16171-d8b0-458e-9ad3-68f85c0d25dc\") "
Sep 10 13:39:14.824549 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:14.824517    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb16171-d8b0-458e-9ad3-68f85c0d25dc-kube-api-access-2pjv6" (OuterVolumeSpecName: "kube-api-access-2pjv6") pod "7eb16171-d8b0-458e-9ad3-68f85c0d25dc" (UID: "7eb16171-d8b0-458e-9ad3-68f85c0d25dc"). InnerVolumeSpecName "kube-api-access-2pjv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:39:14.923615 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:14.923575    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pjv6\" (UniqueName: \"kubernetes.io/projected/7eb16171-d8b0-458e-9ad3-68f85c0d25dc-kube-api-access-2pjv6\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:39:15.513056 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:15.513024    2549 generic.go:356] "Generic (PLEG): container finished" podID="7eb16171-d8b0-458e-9ad3-68f85c0d25dc" containerID="1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f" exitCode=0
Sep 10 13:39:15.513459 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:15.513062    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx"
Sep 10 13:39:15.513459 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:15.513084    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx" event={"ID":"7eb16171-d8b0-458e-9ad3-68f85c0d25dc","Type":"ContainerDied","Data":"1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f"}
Sep 10 13:39:15.513459 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:15.513115    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-8hwlx" event={"ID":"7eb16171-d8b0-458e-9ad3-68f85c0d25dc","Type":"ContainerDied","Data":"1cfc4360cfa350f088b002dcf0c7d17b88084e4f31bceb9131878364eddf4bab"}
Sep 10 13:39:15.513459 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:15.513131    2549 scope.go:117] "RemoveContainer" containerID="1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f"
Sep 10 13:39:15.521284 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:15.521265    2549 scope.go:117] "RemoveContainer" containerID="1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f"
Sep 10 13:39:15.521544 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:39:15.521525    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f\": container with ID starting with 1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f not found: ID does not exist" containerID="1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f"
Sep 10 13:39:15.521588 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:15.521553    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f"} err="failed to get container status \"1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f\": rpc error: code = NotFound desc = could not find container \"1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f\": container with ID starting with 1f0c351eea225017114e2ad5a2fd8a701299a5503d2173941db10c917cda3f2f not found: ID does not exist"
Sep 10 13:39:15.534651 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:15.534626    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8hwlx"]
Sep 10 13:39:15.541023 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:15.541005    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8hwlx"]
Sep 10 13:39:15.672912 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:15.672880    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eb16171-d8b0-458e-9ad3-68f85c0d25dc" path="/var/lib/kubelet/pods/7eb16171-d8b0-458e-9ad3-68f85c0d25dc/volumes"
Sep 10 13:39:38.334893 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.334797    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-gq67g"]
Sep 10 13:39:38.335253 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.335059    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb16171-d8b0-458e-9ad3-68f85c0d25dc" containerName="authorino"
Sep 10 13:39:38.335253 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.335071    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb16171-d8b0-458e-9ad3-68f85c0d25dc" containerName="authorino"
Sep 10 13:39:38.335253 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.335134    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eb16171-d8b0-458e-9ad3-68f85c0d25dc" containerName="authorino"
Sep 10 13:39:38.337620 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.337605    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gq67g"
Sep 10 13:39:38.348437 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.348415    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-gq67g"]
Sep 10 13:39:38.395203 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.395178    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-gq67g"]
Sep 10 13:39:38.395393 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:39:38.395376    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7nxkj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-gq67g" podUID="e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2"
Sep 10 13:39:38.396356 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.396324    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nxkj\" (UniqueName: \"kubernetes.io/projected/e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2-kube-api-access-7nxkj\") pod \"authorino-557f876f69-gq67g\" (UID: \"e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2\") " pod="kuadrant-system/authorino-557f876f69-gq67g"
Sep 10 13:39:38.425276 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.425250    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-bwf28"]
Sep 10 13:39:38.428023 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.428007    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-bwf28"
Sep 10 13:39:38.430387 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.430365    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 13:39:38.439039 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.439020    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-bwf28"]
Sep 10 13:39:38.496830 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.496803    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/72771707-5600-4c9f-a3ef-ea23774f3079-tls-cert\") pod \"authorino-6d4dfc649f-bwf28\" (UID: \"72771707-5600-4c9f-a3ef-ea23774f3079\") " pod="kuadrant-system/authorino-6d4dfc649f-bwf28"
Sep 10 13:39:38.496969 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.496855    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nxkj\" (UniqueName: \"kubernetes.io/projected/e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2-kube-api-access-7nxkj\") pod \"authorino-557f876f69-gq67g\" (UID: \"e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2\") " pod="kuadrant-system/authorino-557f876f69-gq67g"
Sep 10 13:39:38.496969 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.496931    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgvz8\" (UniqueName: \"kubernetes.io/projected/72771707-5600-4c9f-a3ef-ea23774f3079-kube-api-access-xgvz8\") pod \"authorino-6d4dfc649f-bwf28\" (UID: \"72771707-5600-4c9f-a3ef-ea23774f3079\") " pod="kuadrant-system/authorino-6d4dfc649f-bwf28"
Sep 10 13:39:38.506856 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.506825    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nxkj\" (UniqueName: \"kubernetes.io/projected/e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2-kube-api-access-7nxkj\") pod \"authorino-557f876f69-gq67g\" (UID: \"e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2\") " pod="kuadrant-system/authorino-557f876f69-gq67g"
Sep 10 13:39:38.586734 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.586661    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-bwf28"]
Sep 10 13:39:38.586947 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:39:38.586924    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xgvz8 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-bwf28" podUID="72771707-5600-4c9f-a3ef-ea23774f3079"
Sep 10 13:39:38.597709 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.597686    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgvz8\" (UniqueName: \"kubernetes.io/projected/72771707-5600-4c9f-a3ef-ea23774f3079-kube-api-access-xgvz8\") pod \"authorino-6d4dfc649f-bwf28\" (UID: \"72771707-5600-4c9f-a3ef-ea23774f3079\") " pod="kuadrant-system/authorino-6d4dfc649f-bwf28"
Sep 10 13:39:38.597819 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.597743    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/72771707-5600-4c9f-a3ef-ea23774f3079-tls-cert\") pod \"authorino-6d4dfc649f-bwf28\" (UID: \"72771707-5600-4c9f-a3ef-ea23774f3079\") " pod="kuadrant-system/authorino-6d4dfc649f-bwf28"
Sep 10 13:39:38.598365 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.598343    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-bwf28"
Sep 10 13:39:38.598523 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.598509    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gq67g"
Sep 10 13:39:38.600089 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.600071    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/72771707-5600-4c9f-a3ef-ea23774f3079-tls-cert\") pod \"authorino-6d4dfc649f-bwf28\" (UID: \"72771707-5600-4c9f-a3ef-ea23774f3079\") " pod="kuadrant-system/authorino-6d4dfc649f-bwf28"
Sep 10 13:39:38.610780 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.610757    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgvz8\" (UniqueName: \"kubernetes.io/projected/72771707-5600-4c9f-a3ef-ea23774f3079-kube-api-access-xgvz8\") pod \"authorino-6d4dfc649f-bwf28\" (UID: \"72771707-5600-4c9f-a3ef-ea23774f3079\") " pod="kuadrant-system/authorino-6d4dfc649f-bwf28"
Sep 10 13:39:38.611479 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.611463    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-bwf28"
Sep 10 13:39:38.620133 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.620111    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gq67g"
Sep 10 13:39:38.623142 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.623122    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9d9bf4f55-jmzjq"]
Sep 10 13:39:38.625700 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.625685    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d9bf4f55-jmzjq"
Sep 10 13:39:38.636338 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.636318    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9d9bf4f55-jmzjq"]
Sep 10 13:39:38.698051 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.698023    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/72771707-5600-4c9f-a3ef-ea23774f3079-tls-cert\") pod \"72771707-5600-4c9f-a3ef-ea23774f3079\" (UID: \"72771707-5600-4c9f-a3ef-ea23774f3079\") "
Sep 10 13:39:38.698206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.698066    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgvz8\" (UniqueName: \"kubernetes.io/projected/72771707-5600-4c9f-a3ef-ea23774f3079-kube-api-access-xgvz8\") pod \"72771707-5600-4c9f-a3ef-ea23774f3079\" (UID: \"72771707-5600-4c9f-a3ef-ea23774f3079\") "
Sep 10 13:39:38.698206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.698101    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nxkj\" (UniqueName: \"kubernetes.io/projected/e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2-kube-api-access-7nxkj\") pod \"e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2\" (UID: \"e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2\") "
Sep 10 13:39:38.698206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.698188    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f65886a-8737-4535-9af3-784551366125-tls-cert\") pod \"authorino-9d9bf4f55-jmzjq\" (UID: \"9f65886a-8737-4535-9af3-784551366125\") " pod="kuadrant-system/authorino-9d9bf4f55-jmzjq"
Sep 10 13:39:38.698360 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.698218    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7wdl\" (UniqueName: \"kubernetes.io/projected/9f65886a-8737-4535-9af3-784551366125-kube-api-access-s7wdl\") pod \"authorino-9d9bf4f55-jmzjq\" (UID: \"9f65886a-8737-4535-9af3-784551366125\") " pod="kuadrant-system/authorino-9d9bf4f55-jmzjq"
Sep 10 13:39:38.700201 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.700174    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72771707-5600-4c9f-a3ef-ea23774f3079-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "72771707-5600-4c9f-a3ef-ea23774f3079" (UID: "72771707-5600-4c9f-a3ef-ea23774f3079"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:39:38.700201 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.700195    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2-kube-api-access-7nxkj" (OuterVolumeSpecName: "kube-api-access-7nxkj") pod "e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2" (UID: "e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2"). InnerVolumeSpecName "kube-api-access-7nxkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:39:38.700338 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.700198    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72771707-5600-4c9f-a3ef-ea23774f3079-kube-api-access-xgvz8" (OuterVolumeSpecName: "kube-api-access-xgvz8") pod "72771707-5600-4c9f-a3ef-ea23774f3079" (UID: "72771707-5600-4c9f-a3ef-ea23774f3079"). InnerVolumeSpecName "kube-api-access-xgvz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:39:38.798682 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.798649    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f65886a-8737-4535-9af3-784551366125-tls-cert\") pod \"authorino-9d9bf4f55-jmzjq\" (UID: \"9f65886a-8737-4535-9af3-784551366125\") " pod="kuadrant-system/authorino-9d9bf4f55-jmzjq"
Sep 10 13:39:38.798822 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.798694    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7wdl\" (UniqueName: \"kubernetes.io/projected/9f65886a-8737-4535-9af3-784551366125-kube-api-access-s7wdl\") pod \"authorino-9d9bf4f55-jmzjq\" (UID: \"9f65886a-8737-4535-9af3-784551366125\") " pod="kuadrant-system/authorino-9d9bf4f55-jmzjq"
Sep 10 13:39:38.798822 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.798740    2549 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/72771707-5600-4c9f-a3ef-ea23774f3079-tls-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:39:38.798822 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.798752    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgvz8\" (UniqueName: \"kubernetes.io/projected/72771707-5600-4c9f-a3ef-ea23774f3079-kube-api-access-xgvz8\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:39:38.798822 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.798762    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nxkj\" (UniqueName: \"kubernetes.io/projected/e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2-kube-api-access-7nxkj\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:39:38.800833 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.800805    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f65886a-8737-4535-9af3-784551366125-tls-cert\") pod \"authorino-9d9bf4f55-jmzjq\" (UID: \"9f65886a-8737-4535-9af3-784551366125\") " pod="kuadrant-system/authorino-9d9bf4f55-jmzjq"
Sep 10 13:39:38.808896 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.808858    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7wdl\" (UniqueName: \"kubernetes.io/projected/9f65886a-8737-4535-9af3-784551366125-kube-api-access-s7wdl\") pod \"authorino-9d9bf4f55-jmzjq\" (UID: \"9f65886a-8737-4535-9af3-784551366125\") " pod="kuadrant-system/authorino-9d9bf4f55-jmzjq"
Sep 10 13:39:38.933545 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:38.933517    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d9bf4f55-jmzjq"
Sep 10 13:39:39.204132 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:39.204058    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9d9bf4f55-jmzjq"]
Sep 10 13:39:39.207299 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:39:39.207270    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f65886a_8737_4535_9af3_784551366125.slice/crio-ae94068260e7f4563933450db3f05efde422d1b7563eeb7d3e4c81bf4a363a08 WatchSource:0}: Error finding container ae94068260e7f4563933450db3f05efde422d1b7563eeb7d3e4c81bf4a363a08: Status 404 returned error can't find the container with id ae94068260e7f4563933450db3f05efde422d1b7563eeb7d3e4c81bf4a363a08
Sep 10 13:39:39.602905 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:39.602805    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gq67g"
Sep 10 13:39:39.602905 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:39.602815    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d9bf4f55-jmzjq" event={"ID":"9f65886a-8737-4535-9af3-784551366125","Type":"ContainerStarted","Data":"ae94068260e7f4563933450db3f05efde422d1b7563eeb7d3e4c81bf4a363a08"}
Sep 10 13:39:39.603375 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:39.602981    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-bwf28"
Sep 10 13:39:39.636834 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:39.636798    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-bwf28"]
Sep 10 13:39:39.643675 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:39.643648    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-bwf28"]
Sep 10 13:39:39.672809 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:39.672782    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72771707-5600-4c9f-a3ef-ea23774f3079" path="/var/lib/kubelet/pods/72771707-5600-4c9f-a3ef-ea23774f3079/volumes"
Sep 10 13:39:39.672964 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:39.672949    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-gq67g"]
Sep 10 13:39:39.674607 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:39.674587    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-gq67g"]
Sep 10 13:39:40.607704 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:40.607667    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d9bf4f55-jmzjq" event={"ID":"9f65886a-8737-4535-9af3-784551366125","Type":"ContainerStarted","Data":"35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be"}
Sep 10 13:39:40.627849 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:40.627807    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9d9bf4f55-jmzjq" podStartSLOduration=1.841065177 podStartE2EDuration="2.627790446s" totalImagesPullingTime="786.725269ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:40.626401418 +0000 UTC m=+539.532959449" watchObservedRunningTime="2026-09-10 13:39:40.627790446 +0000 UTC m=+539.534348478"
Sep 10 13:39:40.658699 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:40.658672    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-fr28j"]
Sep 10 13:39:40.658912 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:40.658886    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-fr28j" podUID="d16cc6e9-aa23-458d-b29f-ffb01e0e6204" containerName="authorino" containerID="cri-o://b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664" gracePeriod=30
Sep 10 13:39:40.899301 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:40.899279    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-fr28j"
Sep 10 13:39:41.015503 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.015477    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6vjz\" (UniqueName: \"kubernetes.io/projected/d16cc6e9-aa23-458d-b29f-ffb01e0e6204-kube-api-access-b6vjz\") pod \"d16cc6e9-aa23-458d-b29f-ffb01e0e6204\" (UID: \"d16cc6e9-aa23-458d-b29f-ffb01e0e6204\") "
Sep 10 13:39:41.017478 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.017445    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d16cc6e9-aa23-458d-b29f-ffb01e0e6204-kube-api-access-b6vjz" (OuterVolumeSpecName: "kube-api-access-b6vjz") pod "d16cc6e9-aa23-458d-b29f-ffb01e0e6204" (UID: "d16cc6e9-aa23-458d-b29f-ffb01e0e6204"). InnerVolumeSpecName "kube-api-access-b6vjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:39:41.116928 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.116898    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6vjz\" (UniqueName: \"kubernetes.io/projected/d16cc6e9-aa23-458d-b29f-ffb01e0e6204-kube-api-access-b6vjz\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:39:41.611564 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.611531    2549 generic.go:356] "Generic (PLEG): container finished" podID="d16cc6e9-aa23-458d-b29f-ffb01e0e6204" containerID="b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664" exitCode=0
Sep 10 13:39:41.611980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.611568    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-fr28j"
Sep 10 13:39:41.611980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.611609    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-fr28j" event={"ID":"d16cc6e9-aa23-458d-b29f-ffb01e0e6204","Type":"ContainerDied","Data":"b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664"}
Sep 10 13:39:41.611980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.611644    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-fr28j" event={"ID":"d16cc6e9-aa23-458d-b29f-ffb01e0e6204","Type":"ContainerDied","Data":"a976474b93b5dfeb9346f9b84e5fb131ae637c0b8bea32f2e4d8a6c32177cce3"}
Sep 10 13:39:41.611980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.611662    2549 scope.go:117] "RemoveContainer" containerID="b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664"
Sep 10 13:39:41.619707 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.619690    2549 scope.go:117] "RemoveContainer" containerID="b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664"
Sep 10 13:39:41.620832 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:39:41.620804    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664\": container with ID starting with b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664 not found: ID does not exist" containerID="b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664"
Sep 10 13:39:41.620950 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.620841    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664"} err="failed to get container status \"b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664\": rpc error: code = NotFound desc = could not find container \"b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664\": container with ID starting with b1bd5aebd17d0cf2900234297df7a8b20b2357f3a7474c8c2ec7dfd92a7cc664 not found: ID does not exist"
Sep 10 13:39:41.639874 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.639841    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-fr28j"]
Sep 10 13:39:41.649441 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.649420    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-fr28j"]
Sep 10 13:39:41.672060 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.672040    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d16cc6e9-aa23-458d-b29f-ffb01e0e6204" path="/var/lib/kubelet/pods/d16cc6e9-aa23-458d-b29f-ffb01e0e6204/volumes"
Sep 10 13:39:41.672352 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:41.672340    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2" path="/var/lib/kubelet/pods/e9e14bcd-8ebb-4b12-a46e-c95f05c33fc2/volumes"
Sep 10 13:39:45.927264 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:45.927229    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-54958d54c6-p9hfd"]
Sep 10 13:39:45.927609 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:45.927513    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d16cc6e9-aa23-458d-b29f-ffb01e0e6204" containerName="authorino"
Sep 10 13:39:45.927609 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:45.927525    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d16cc6e9-aa23-458d-b29f-ffb01e0e6204" containerName="authorino"
Sep 10 13:39:45.927609 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:45.927594    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="d16cc6e9-aa23-458d-b29f-ffb01e0e6204" containerName="authorino"
Sep 10 13:39:45.931775 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:45.931759    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:45.936431 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:45.936390    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lz67j\""
Sep 10 13:39:45.936557 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:45.936508    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 13:39:45.936683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:45.936659    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 13:39:45.937419 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:45.937402    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 13:39:45.943478 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:45.943459    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54958d54c6-p9hfd"]
Sep 10 13:39:46.053755 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.053727    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-metrics-certs\") pod \"maas-controller-54958d54c6-p9hfd\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") " pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:46.053910 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.053765    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-webhook-cert\") pod \"maas-controller-54958d54c6-p9hfd\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") " pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:46.053910 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.053788    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvlz8\" (UniqueName: \"kubernetes.io/projected/3f709f70-004a-4cec-ad57-9b5a69ad0e13-kube-api-access-qvlz8\") pod \"maas-controller-54958d54c6-p9hfd\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") " pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:46.154915 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.154885    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-webhook-cert\") pod \"maas-controller-54958d54c6-p9hfd\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") " pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:46.155039 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.154923    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvlz8\" (UniqueName: \"kubernetes.io/projected/3f709f70-004a-4cec-ad57-9b5a69ad0e13-kube-api-access-qvlz8\") pod \"maas-controller-54958d54c6-p9hfd\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") " pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:46.155039 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.154993    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-metrics-certs\") pod \"maas-controller-54958d54c6-p9hfd\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") " pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:46.155122 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:39:46.155048    2549 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 10 13:39:46.155122 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:39:46.155118    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-webhook-cert podName:3f709f70-004a-4cec-ad57-9b5a69ad0e13 nodeName:}" failed. No retries permitted until 2026-09-10 13:39:46.655097276 +0000 UTC m=+545.561655290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-webhook-cert") pod "maas-controller-54958d54c6-p9hfd" (UID: "3f709f70-004a-4cec-ad57-9b5a69ad0e13") : secret "maas-controller-webhook-cert" not found
Sep 10 13:39:46.157396 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.157371    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-metrics-certs\") pod \"maas-controller-54958d54c6-p9hfd\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") " pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:46.174461 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.174436    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvlz8\" (UniqueName: \"kubernetes.io/projected/3f709f70-004a-4cec-ad57-9b5a69ad0e13-kube-api-access-qvlz8\") pod \"maas-controller-54958d54c6-p9hfd\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") " pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:46.659804 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.659767    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-webhook-cert\") pod \"maas-controller-54958d54c6-p9hfd\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") " pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:46.662190 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.662154    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-webhook-cert\") pod \"maas-controller-54958d54c6-p9hfd\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") " pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:46.841386 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:46.841334    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:47.120249 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:47.120232    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54958d54c6-p9hfd"]
Sep 10 13:39:47.122447 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:39:47.122421    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f709f70_004a_4cec_ad57_9b5a69ad0e13.slice/crio-72a6f31a663ce26e0b2c3cfb6eb3758ec8e85ff2f83f7823e9c802ef42fdcfe9 WatchSource:0}: Error finding container 72a6f31a663ce26e0b2c3cfb6eb3758ec8e85ff2f83f7823e9c802ef42fdcfe9: Status 404 returned error can't find the container with id 72a6f31a663ce26e0b2c3cfb6eb3758ec8e85ff2f83f7823e9c802ef42fdcfe9
Sep 10 13:39:47.636445 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:47.636408    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54958d54c6-p9hfd" event={"ID":"3f709f70-004a-4cec-ad57-9b5a69ad0e13","Type":"ContainerStarted","Data":"72a6f31a663ce26e0b2c3cfb6eb3758ec8e85ff2f83f7823e9c802ef42fdcfe9"}
Sep 10 13:39:49.645896 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:49.645777    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54958d54c6-p9hfd" event={"ID":"3f709f70-004a-4cec-ad57-9b5a69ad0e13","Type":"ContainerStarted","Data":"f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34"}
Sep 10 13:39:49.646336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:49.645927    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:39:49.666719 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:39:49.666675    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-54958d54c6-p9hfd" podStartSLOduration=2.449136019 podStartE2EDuration="4.66666155s" totalImagesPullingTime="2.217525531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:49.666508313 +0000 UTC m=+548.573066360" watchObservedRunningTime="2026-09-10 13:39:49.66666155 +0000 UTC m=+548.573219629"
Sep 10 13:40:00.651568 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:00.651534    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:40:01.040654 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.040623    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54958d54c6-p9hfd"]
Sep 10 13:40:01.040878 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.040838    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-54958d54c6-p9hfd" podUID="3f709f70-004a-4cec-ad57-9b5a69ad0e13" containerName="manager" containerID="cri-o://f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34" gracePeriod=30
Sep 10 13:40:01.281956 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.281930    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:40:01.477537 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.477502    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-metrics-certs\") pod \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") "
Sep 10 13:40:01.477680 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.477574    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvlz8\" (UniqueName: \"kubernetes.io/projected/3f709f70-004a-4cec-ad57-9b5a69ad0e13-kube-api-access-qvlz8\") pod \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") "
Sep 10 13:40:01.477680 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.477603    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-webhook-cert\") pod \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\" (UID: \"3f709f70-004a-4cec-ad57-9b5a69ad0e13\") "
Sep 10 13:40:01.479710 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.479676    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3f709f70-004a-4cec-ad57-9b5a69ad0e13" (UID: "3f709f70-004a-4cec-ad57-9b5a69ad0e13"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:40:01.479710 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.479697    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3f709f70-004a-4cec-ad57-9b5a69ad0e13" (UID: "3f709f70-004a-4cec-ad57-9b5a69ad0e13"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:40:01.479857 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.479688    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f709f70-004a-4cec-ad57-9b5a69ad0e13-kube-api-access-qvlz8" (OuterVolumeSpecName: "kube-api-access-qvlz8") pod "3f709f70-004a-4cec-ad57-9b5a69ad0e13" (UID: "3f709f70-004a-4cec-ad57-9b5a69ad0e13"). InnerVolumeSpecName "kube-api-access-qvlz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:40:01.578904 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.578851    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-metrics-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:40:01.578904 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.578898    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvlz8\" (UniqueName: \"kubernetes.io/projected/3f709f70-004a-4cec-ad57-9b5a69ad0e13-kube-api-access-qvlz8\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:40:01.579064 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.578912    2549 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f709f70-004a-4cec-ad57-9b5a69ad0e13-webhook-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:40:01.690018 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.689991    2549 generic.go:356] "Generic (PLEG): container finished" podID="3f709f70-004a-4cec-ad57-9b5a69ad0e13" containerID="f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34" exitCode=0
Sep 10 13:40:01.690346 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.690035    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54958d54c6-p9hfd"
Sep 10 13:40:01.690346 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.690077    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54958d54c6-p9hfd" event={"ID":"3f709f70-004a-4cec-ad57-9b5a69ad0e13","Type":"ContainerDied","Data":"f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34"}
Sep 10 13:40:01.690346 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.690128    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54958d54c6-p9hfd" event={"ID":"3f709f70-004a-4cec-ad57-9b5a69ad0e13","Type":"ContainerDied","Data":"72a6f31a663ce26e0b2c3cfb6eb3758ec8e85ff2f83f7823e9c802ef42fdcfe9"}
Sep 10 13:40:01.690346 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.690151    2549 scope.go:117] "RemoveContainer" containerID="f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34"
Sep 10 13:40:01.697638 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.697616    2549 scope.go:117] "RemoveContainer" containerID="f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34"
Sep 10 13:40:01.697931 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:40:01.697911    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34\": container with ID starting with f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34 not found: ID does not exist" containerID="f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34"
Sep 10 13:40:01.697981 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.697940    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34"} err="failed to get container status \"f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34\": rpc error: code = NotFound desc = could not find container \"f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34\": container with ID starting with f1b08bf77f66b6f2cfd3eed4ec4fa66c56f4e6fb144dea63dec87ba865d39d34 not found: ID does not exist"
Sep 10 13:40:01.722017 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.721988    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54958d54c6-p9hfd"]
Sep 10 13:40:01.723961 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:01.723942    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-54958d54c6-p9hfd"]
Sep 10 13:40:03.673002 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:03.672970    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f709f70-004a-4cec-ad57-9b5a69ad0e13" path="/var/lib/kubelet/pods/3f709f70-004a-4cec-ad57-9b5a69ad0e13/volumes"
Sep 10 13:40:23.439646 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.439604    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"]
Sep 10 13:40:23.440199 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.439904    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f709f70-004a-4cec-ad57-9b5a69ad0e13" containerName="manager"
Sep 10 13:40:23.440199 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.439916    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f709f70-004a-4cec-ad57-9b5a69ad0e13" containerName="manager"
Sep 10 13:40:23.440199 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.439969    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f709f70-004a-4cec-ad57-9b5a69ad0e13" containerName="manager"
Sep 10 13:40:23.446443 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.446415    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"
Sep 10 13:40:23.449456 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.449432    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 13:40:23.449580 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.449524    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5bdb5\""
Sep 10 13:40:23.456213 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.456174    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"]
Sep 10 13:40:23.543000 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.542965    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc4baee-8cc6-4af3-bf61-dd80ed85b541-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-rvwwf\" (UID: \"4dc4baee-8cc6-4af3-bf61-dd80ed85b541\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"
Sep 10 13:40:23.543164 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.543032    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxjlb\" (UniqueName: \"kubernetes.io/projected/4dc4baee-8cc6-4af3-bf61-dd80ed85b541-kube-api-access-lxjlb\") pod \"payload-pre-processing-59fbc47bf8-rvwwf\" (UID: \"4dc4baee-8cc6-4af3-bf61-dd80ed85b541\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"
Sep 10 13:40:23.643995 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.643962    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxjlb\" (UniqueName: \"kubernetes.io/projected/4dc4baee-8cc6-4af3-bf61-dd80ed85b541-kube-api-access-lxjlb\") pod \"payload-pre-processing-59fbc47bf8-rvwwf\" (UID: \"4dc4baee-8cc6-4af3-bf61-dd80ed85b541\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"
Sep 10 13:40:23.644177 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.644022    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc4baee-8cc6-4af3-bf61-dd80ed85b541-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-rvwwf\" (UID: \"4dc4baee-8cc6-4af3-bf61-dd80ed85b541\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"
Sep 10 13:40:23.644635 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.644616    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc4baee-8cc6-4af3-bf61-dd80ed85b541-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-rvwwf\" (UID: \"4dc4baee-8cc6-4af3-bf61-dd80ed85b541\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"
Sep 10 13:40:23.660048 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.659997    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxjlb\" (UniqueName: \"kubernetes.io/projected/4dc4baee-8cc6-4af3-bf61-dd80ed85b541-kube-api-access-lxjlb\") pod \"payload-pre-processing-59fbc47bf8-rvwwf\" (UID: \"4dc4baee-8cc6-4af3-bf61-dd80ed85b541\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"
Sep 10 13:40:23.759583 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:23.759505    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"
Sep 10 13:40:24.034675 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:24.034604    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"]
Sep 10 13:40:24.038002 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:40:24.037973    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc4baee_8cc6_4af3_bf61_dd80ed85b541.slice/crio-35b6080a6c3f2dbef70cbccbc0eaf557dfc93f219bb5754a05dfe356ca1be070 WatchSource:0}: Error finding container 35b6080a6c3f2dbef70cbccbc0eaf557dfc93f219bb5754a05dfe356ca1be070: Status 404 returned error can't find the container with id 35b6080a6c3f2dbef70cbccbc0eaf557dfc93f219bb5754a05dfe356ca1be070
Sep 10 13:40:24.770689 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:24.770657    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf" event={"ID":"4dc4baee-8cc6-4af3-bf61-dd80ed85b541","Type":"ContainerStarted","Data":"35b6080a6c3f2dbef70cbccbc0eaf557dfc93f219bb5754a05dfe356ca1be070"}
Sep 10 13:40:26.781203 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:26.781165    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf" event={"ID":"4dc4baee-8cc6-4af3-bf61-dd80ed85b541","Type":"ContainerStarted","Data":"b0c0a813aa4e093f26c2ed7152eb2ef22987a49a76b581cf279e7e789f019725"}
Sep 10 13:40:26.781646 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:26.781275    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"
Sep 10 13:40:37.786856 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:37.786822    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf"
Sep 10 13:40:37.807848 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:37.807802    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rvwwf" podStartSLOduration=12.268332906 podStartE2EDuration="14.807787912s" totalImagesPullingTime="2.539455006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:26.807547863 +0000 UTC m=+585.714105894" watchObservedRunningTime="2026-09-10 13:40:37.807787912 +0000 UTC m=+596.714345943"
Sep 10 13:40:41.508481 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:41.508450    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:40:41.508895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:40:41.508662    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:41:16.102195 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.102163    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"]
Sep 10 13:41:16.104047 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.104033    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.107298 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.107279    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zm5zz\""
Sep 10 13:41:16.108280 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.108265    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 13:41:16.108339 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.108309    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 13:41:16.108384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.108314    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 13:41:16.119458 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.119430    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"]
Sep 10 13:41:16.236980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.236946    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.236980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.236984    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.237184 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.237011    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.237184 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.237069    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.237184 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.237114    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358c4045-aa79-4c46-9c39-2dc5c1935411-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.237184 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.237174    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv74n\" (UniqueName: \"kubernetes.io/projected/358c4045-aa79-4c46-9c39-2dc5c1935411-kube-api-access-hv74n\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.338514 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.338473    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv74n\" (UniqueName: \"kubernetes.io/projected/358c4045-aa79-4c46-9c39-2dc5c1935411-kube-api-access-hv74n\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.338701 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.338527    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.338701 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.338551    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.338701 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.338568    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.338701 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.338598    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.338701 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.338616    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358c4045-aa79-4c46-9c39-2dc5c1935411-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.339020 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.339001    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.339112 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.339088    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.339166 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.339108    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.340815 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.340797    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358c4045-aa79-4c46-9c39-2dc5c1935411-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.341229 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.341208    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358c4045-aa79-4c46-9c39-2dc5c1935411-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.349066 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.349038    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv74n\" (UniqueName: \"kubernetes.io/projected/358c4045-aa79-4c46-9c39-2dc5c1935411-kube-api-access-hv74n\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr\" (UID: \"358c4045-aa79-4c46-9c39-2dc5c1935411\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.413305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.413274    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:16.689602 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.689577    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"]
Sep 10 13:41:16.692169 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:41:16.692142    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358c4045_aa79_4c46_9c39_2dc5c1935411.slice/crio-b533ecf5c9dfad85248ed63e3985296496802ec323b8d558722be4636ba00a2f WatchSource:0}: Error finding container b533ecf5c9dfad85248ed63e3985296496802ec323b8d558722be4636ba00a2f: Status 404 returned error can't find the container with id b533ecf5c9dfad85248ed63e3985296496802ec323b8d558722be4636ba00a2f
Sep 10 13:41:16.693391 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.693371    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:41:16.960007 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:16.959928    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr" event={"ID":"358c4045-aa79-4c46-9c39-2dc5c1935411","Type":"ContainerStarted","Data":"b533ecf5c9dfad85248ed63e3985296496802ec323b8d558722be4636ba00a2f"}
Sep 10 13:41:17.965373 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:17.965285    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr" event={"ID":"358c4045-aa79-4c46-9c39-2dc5c1935411","Type":"ContainerStarted","Data":"0941d821257a0af234a1063e931685cf0d3c6cb4f7e148a625371343f3406bfd"}
Sep 10 13:41:18.003430 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:18.003382    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr" podStartSLOduration=0.990778745 podStartE2EDuration="2.003366671s" totalImagesPullingTime="1.012587926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:18.001977598 +0000 UTC m=+636.908535629" watchObservedRunningTime="2026-09-10 13:41:18.003366671 +0000 UTC m=+636.909924702"
Sep 10 13:41:26.414206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:26.414168    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:26.414612 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:26.414242    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:26.426657 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:26.426625    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:27.008936 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:27.008907    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr"
Sep 10 13:41:35.899500 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:35.899468    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"]
Sep 10 13:41:35.901666 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:35.901644    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:35.905408 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:35.905382    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 13:41:35.915572 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:35.915549    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"]
Sep 10 13:41:35.964686 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:35.964656    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0602337-9751-4c54-9255-9fd9b88973e1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:35.964686 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:35.964690    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:35.964892 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:35.964708    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcs59\" (UniqueName: \"kubernetes.io/projected/a0602337-9751-4c54-9255-9fd9b88973e1-kube-api-access-zcs59\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:35.964892 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:35.964796    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:35.964892 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:35.964838    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:35.964892 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:35.964856    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.065998 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.065959    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.065998 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.065998    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.066220 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.066054    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0602337-9751-4c54-9255-9fd9b88973e1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.066220 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.066083    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.066220 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.066105    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcs59\" (UniqueName: \"kubernetes.io/projected/a0602337-9751-4c54-9255-9fd9b88973e1-kube-api-access-zcs59\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.066220 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.066146    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.066527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.066496    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.066527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.066503    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.066687 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.066556    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.068625 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.068602    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0602337-9751-4c54-9255-9fd9b88973e1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.068897 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.068880    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0602337-9751-4c54-9255-9fd9b88973e1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.078363 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.078338    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcs59\" (UniqueName: \"kubernetes.io/projected/a0602337-9751-4c54-9255-9fd9b88973e1-kube-api-access-zcs59\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4q6rk\" (UID: \"a0602337-9751-4c54-9255-9fd9b88973e1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.211006 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.210977    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:36.490641 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:36.490398    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"]
Sep 10 13:41:36.493181 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:41:36.493155    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0602337_9751_4c54_9255_9fd9b88973e1.slice/crio-910ccd8cbfecf12581970317b4a6a5b631f6392c190282aea5691fe6f435bc9d WatchSource:0}: Error finding container 910ccd8cbfecf12581970317b4a6a5b631f6392c190282aea5691fe6f435bc9d: Status 404 returned error can't find the container with id 910ccd8cbfecf12581970317b4a6a5b631f6392c190282aea5691fe6f435bc9d
Sep 10 13:41:37.035629 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:37.035537    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk" event={"ID":"a0602337-9751-4c54-9255-9fd9b88973e1","Type":"ContainerStarted","Data":"c957044726f2a2f2b0ee817c99299bfc5ef21eab1dc9783a485c0829c4115ae8"}
Sep 10 13:41:37.035629 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:37.035597    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk" event={"ID":"a0602337-9751-4c54-9255-9fd9b88973e1","Type":"ContainerStarted","Data":"910ccd8cbfecf12581970317b4a6a5b631f6392c190282aea5691fe6f435bc9d"}
Sep 10 13:41:37.073323 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:37.073263    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk" podStartSLOduration=1.85631748 podStartE2EDuration="2.073241431s" totalImagesPullingTime="216.923951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:37.072728846 +0000 UTC m=+655.979286878" watchObservedRunningTime="2026-09-10 13:41:37.073241431 +0000 UTC m=+655.979799463"
Sep 10 13:41:46.211912 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:46.211847    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:46.212439 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:46.211939    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:46.224491 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:46.224467    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:41:47.085310 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:41:47.085282    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4q6rk"
Sep 10 13:42:09.092007 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.091948    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"]
Sep 10 13:42:09.095084 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.095058    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.097644 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.097618    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 13:42:09.106421 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.106397    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"]
Sep 10 13:42:09.247940 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.247904    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpnnv\" (UniqueName: \"kubernetes.io/projected/9ea64d04-56fc-413f-995a-511db7c72b67-kube-api-access-xpnnv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.247940 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.247940    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.248144 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.248010    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.248144 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.248039    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.248144 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.248063    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.248144 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.248085    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ea64d04-56fc-413f-995a-511db7c72b67-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.348793 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.348717    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.348793 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.348768    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.348942 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.348906    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.348985 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.348942    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ea64d04-56fc-413f-995a-511db7c72b67-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.349023 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.348985    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpnnv\" (UniqueName: \"kubernetes.io/projected/9ea64d04-56fc-413f-995a-511db7c72b67-kube-api-access-xpnnv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.349023 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.349003    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.349202 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.349183    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.349251 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.349205    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.349289 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.349258    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.351233 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.351209    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ea64d04-56fc-413f-995a-511db7c72b67-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.351513 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.351493    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ea64d04-56fc-413f-995a-511db7c72b67-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.358659 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.358638    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpnnv\" (UniqueName: \"kubernetes.io/projected/9ea64d04-56fc-413f-995a-511db7c72b67-kube-api-access-xpnnv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4\" (UID: \"9ea64d04-56fc-413f-995a-511db7c72b67\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.403589 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.403560    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:09.680385 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:09.680363    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"]
Sep 10 13:42:09.682781 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:42:09.682757    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ea64d04_56fc_413f_995a_511db7c72b67.slice/crio-833b50b9b499d415fc170768e394fb6f1ac93dec597795f75e476ad98ad2b353 WatchSource:0}: Error finding container 833b50b9b499d415fc170768e394fb6f1ac93dec597795f75e476ad98ad2b353: Status 404 returned error can't find the container with id 833b50b9b499d415fc170768e394fb6f1ac93dec597795f75e476ad98ad2b353
Sep 10 13:42:10.169341 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:10.169306    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4" event={"ID":"9ea64d04-56fc-413f-995a-511db7c72b67","Type":"ContainerStarted","Data":"e9657fbb3a834abb72323eeac23fac3320ad82e2e402eae9a2dbb155de4f719e"}
Sep 10 13:42:10.169341 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:10.169347    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4" event={"ID":"9ea64d04-56fc-413f-995a-511db7c72b67","Type":"ContainerStarted","Data":"833b50b9b499d415fc170768e394fb6f1ac93dec597795f75e476ad98ad2b353"}
Sep 10 13:42:10.189460 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:10.189374    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4" podStartSLOduration=0.975794137 podStartE2EDuration="1.18933828s" totalImagesPullingTime="213.544143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:42:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:42:10.18845044 +0000 UTC m=+689.095008471" watchObservedRunningTime="2026-09-10 13:42:10.18933828 +0000 UTC m=+689.095896310"
Sep 10 13:42:19.403707 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:19.403654    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:19.404253 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:19.403940    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:19.416180 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:19.416155    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:42:20.221482 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:42:20.221452    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4"
Sep 10 13:43:07.723142 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:07.723108    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7477b75cb6-lnfdk"]
Sep 10 13:43:07.726129 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:07.726104    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7477b75cb6-lnfdk"
Sep 10 13:43:07.735537 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:07.735515    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7477b75cb6-lnfdk"]
Sep 10 13:43:07.823646 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:07.823620    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-466q2\" (UniqueName: \"kubernetes.io/projected/318acfbd-8a1d-487a-9548-9ca5808e446f-kube-api-access-466q2\") pod \"authorino-7477b75cb6-lnfdk\" (UID: \"318acfbd-8a1d-487a-9548-9ca5808e446f\") " pod="kuadrant-system/authorino-7477b75cb6-lnfdk"
Sep 10 13:43:07.823787 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:07.823658    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/318acfbd-8a1d-487a-9548-9ca5808e446f-tls-cert\") pod \"authorino-7477b75cb6-lnfdk\" (UID: \"318acfbd-8a1d-487a-9548-9ca5808e446f\") " pod="kuadrant-system/authorino-7477b75cb6-lnfdk"
Sep 10 13:43:07.924209 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:07.924180    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-466q2\" (UniqueName: \"kubernetes.io/projected/318acfbd-8a1d-487a-9548-9ca5808e446f-kube-api-access-466q2\") pod \"authorino-7477b75cb6-lnfdk\" (UID: \"318acfbd-8a1d-487a-9548-9ca5808e446f\") " pod="kuadrant-system/authorino-7477b75cb6-lnfdk"
Sep 10 13:43:07.924367 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:07.924221    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/318acfbd-8a1d-487a-9548-9ca5808e446f-tls-cert\") pod \"authorino-7477b75cb6-lnfdk\" (UID: \"318acfbd-8a1d-487a-9548-9ca5808e446f\") " pod="kuadrant-system/authorino-7477b75cb6-lnfdk"
Sep 10 13:43:07.926635 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:07.926611    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/318acfbd-8a1d-487a-9548-9ca5808e446f-tls-cert\") pod \"authorino-7477b75cb6-lnfdk\" (UID: \"318acfbd-8a1d-487a-9548-9ca5808e446f\") " pod="kuadrant-system/authorino-7477b75cb6-lnfdk"
Sep 10 13:43:07.934368 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:07.934347    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-466q2\" (UniqueName: \"kubernetes.io/projected/318acfbd-8a1d-487a-9548-9ca5808e446f-kube-api-access-466q2\") pod \"authorino-7477b75cb6-lnfdk\" (UID: \"318acfbd-8a1d-487a-9548-9ca5808e446f\") " pod="kuadrant-system/authorino-7477b75cb6-lnfdk"
Sep 10 13:43:08.034824 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:08.034740    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7477b75cb6-lnfdk"
Sep 10 13:43:08.303860 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:08.303682    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7477b75cb6-lnfdk"]
Sep 10 13:43:08.306561 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:43:08.306529    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod318acfbd_8a1d_487a_9548_9ca5808e446f.slice/crio-10e3da50747f5481a94cac8819c881132fc5cdff2bd06cea62d7a467952b41a2 WatchSource:0}: Error finding container 10e3da50747f5481a94cac8819c881132fc5cdff2bd06cea62d7a467952b41a2: Status 404 returned error can't find the container with id 10e3da50747f5481a94cac8819c881132fc5cdff2bd06cea62d7a467952b41a2
Sep 10 13:43:08.400859 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:08.400828    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7477b75cb6-lnfdk" event={"ID":"318acfbd-8a1d-487a-9548-9ca5808e446f","Type":"ContainerStarted","Data":"10e3da50747f5481a94cac8819c881132fc5cdff2bd06cea62d7a467952b41a2"}
Sep 10 13:43:10.410227 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.410188    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7477b75cb6-lnfdk" event={"ID":"318acfbd-8a1d-487a-9548-9ca5808e446f","Type":"ContainerStarted","Data":"a1f496c44ba81b9060a44680dcecf81e54ea7eec129e1d1b27a32e91b38d3775"}
Sep 10 13:43:10.435741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.435686    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7477b75cb6-lnfdk" podStartSLOduration=2.130695662 podStartE2EDuration="3.435669145s" totalImagesPullingTime="1.304973483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:10.434926186 +0000 UTC m=+749.341484239" watchObservedRunningTime="2026-09-10 13:43:10.435669145 +0000 UTC m=+749.342227176"
Sep 10 13:43:10.483555 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.483514    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9d9bf4f55-jmzjq"]
Sep 10 13:43:10.483832 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.483761    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-9d9bf4f55-jmzjq" podUID="9f65886a-8737-4535-9af3-784551366125" containerName="authorino" containerID="cri-o://35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be" gracePeriod=30
Sep 10 13:43:10.724554 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.724530    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d9bf4f55-jmzjq"
Sep 10 13:43:10.851486 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.851443    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7wdl\" (UniqueName: \"kubernetes.io/projected/9f65886a-8737-4535-9af3-784551366125-kube-api-access-s7wdl\") pod \"9f65886a-8737-4535-9af3-784551366125\" (UID: \"9f65886a-8737-4535-9af3-784551366125\") "
Sep 10 13:43:10.851668 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.851513    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f65886a-8737-4535-9af3-784551366125-tls-cert\") pod \"9f65886a-8737-4535-9af3-784551366125\" (UID: \"9f65886a-8737-4535-9af3-784551366125\") "
Sep 10 13:43:10.854035 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.854005    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f65886a-8737-4535-9af3-784551366125-kube-api-access-s7wdl" (OuterVolumeSpecName: "kube-api-access-s7wdl") pod "9f65886a-8737-4535-9af3-784551366125" (UID: "9f65886a-8737-4535-9af3-784551366125"). InnerVolumeSpecName "kube-api-access-s7wdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:43:10.861085 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.861055    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f65886a-8737-4535-9af3-784551366125-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9f65886a-8737-4535-9af3-784551366125" (UID: "9f65886a-8737-4535-9af3-784551366125"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:43:10.952566 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.952535    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7wdl\" (UniqueName: \"kubernetes.io/projected/9f65886a-8737-4535-9af3-784551366125-kube-api-access-s7wdl\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:43:10.952566 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:10.952562    2549 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f65886a-8737-4535-9af3-784551366125-tls-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:43:11.415826 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:11.415793    2549 generic.go:356] "Generic (PLEG): container finished" podID="9f65886a-8737-4535-9af3-784551366125" containerID="35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be" exitCode=0
Sep 10 13:43:11.416266 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:11.415844    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d9bf4f55-jmzjq"
Sep 10 13:43:11.416266 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:11.415892    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d9bf4f55-jmzjq" event={"ID":"9f65886a-8737-4535-9af3-784551366125","Type":"ContainerDied","Data":"35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be"}
Sep 10 13:43:11.416266 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:11.415942    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d9bf4f55-jmzjq" event={"ID":"9f65886a-8737-4535-9af3-784551366125","Type":"ContainerDied","Data":"ae94068260e7f4563933450db3f05efde422d1b7563eeb7d3e4c81bf4a363a08"}
Sep 10 13:43:11.416266 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:11.415962    2549 scope.go:117] "RemoveContainer" containerID="35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be"
Sep 10 13:43:11.423897 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:11.423874    2549 scope.go:117] "RemoveContainer" containerID="35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be"
Sep 10 13:43:11.424130 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:43:11.424110    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be\": container with ID starting with 35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be not found: ID does not exist" containerID="35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be"
Sep 10 13:43:11.424195 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:11.424143    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be"} err="failed to get container status \"35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be\": rpc error: code = NotFound desc = could not find container \"35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be\": container with ID starting with 35d977d4f05001393f74fc21f0b07c575d6a681ebad77795bcc26c202d6578be not found: ID does not exist"
Sep 10 13:43:11.463080 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:11.463050    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9d9bf4f55-jmzjq"]
Sep 10 13:43:11.467144 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:11.467122    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-9d9bf4f55-jmzjq"]
Sep 10 13:43:11.673746 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:11.673663    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f65886a-8737-4535-9af3-784551366125" path="/var/lib/kubelet/pods/9f65886a-8737-4535-9af3-784551366125/volumes"
Sep 10 13:43:38.223117 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.223069    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7853-66e934"]
Sep 10 13:43:38.223499 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.223362    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f65886a-8737-4535-9af3-784551366125" containerName="authorino"
Sep 10 13:43:38.223499 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.223373    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f65886a-8737-4535-9af3-784551366125" containerName="authorino"
Sep 10 13:43:38.223499 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.223436    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f65886a-8737-4535-9af3-784551366125" containerName="authorino"
Sep 10 13:43:38.226148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.226133    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7853-66e934"
Sep 10 13:43:38.228739 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.228719    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-g8rj6\""
Sep 10 13:43:38.234082 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.234054    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7853-66e934"]
Sep 10 13:43:38.275347 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.275324    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdsfk\" (UniqueName: \"kubernetes.io/projected/0e5dbd52-786e-4de0-a9be-0a8715e08185-kube-api-access-sdsfk\") pod \"test-curl-7853-66e934\" (UID: \"0e5dbd52-786e-4de0-a9be-0a8715e08185\") " pod="opendatahub/test-curl-7853-66e934"
Sep 10 13:43:38.375670 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.375641    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdsfk\" (UniqueName: \"kubernetes.io/projected/0e5dbd52-786e-4de0-a9be-0a8715e08185-kube-api-access-sdsfk\") pod \"test-curl-7853-66e934\" (UID: \"0e5dbd52-786e-4de0-a9be-0a8715e08185\") " pod="opendatahub/test-curl-7853-66e934"
Sep 10 13:43:38.384901 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.384876    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdsfk\" (UniqueName: \"kubernetes.io/projected/0e5dbd52-786e-4de0-a9be-0a8715e08185-kube-api-access-sdsfk\") pod \"test-curl-7853-66e934\" (UID: \"0e5dbd52-786e-4de0-a9be-0a8715e08185\") " pod="opendatahub/test-curl-7853-66e934"
Sep 10 13:43:38.535364 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.535299    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7853-66e934"
Sep 10 13:43:38.804132 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:38.804059    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7853-66e934"]
Sep 10 13:43:38.807320 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:43:38.807288    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e5dbd52_786e_4de0_a9be_0a8715e08185.slice/crio-5094e8c8ef7e2d530b7825ea75860183204987c8a1504277dfc7d25e73607e48 WatchSource:0}: Error finding container 5094e8c8ef7e2d530b7825ea75860183204987c8a1504277dfc7d25e73607e48: Status 404 returned error can't find the container with id 5094e8c8ef7e2d530b7825ea75860183204987c8a1504277dfc7d25e73607e48
Sep 10 13:43:39.529337 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:39.529298    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7853-66e934" event={"ID":"0e5dbd52-786e-4de0-a9be-0a8715e08185","Type":"ContainerStarted","Data":"5094e8c8ef7e2d530b7825ea75860183204987c8a1504277dfc7d25e73607e48"}
Sep 10 13:43:40.991080 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:40.991051    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"]
Sep 10 13:43:40.994775 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:40.994752    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:43:40.997271 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:40.997245    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-f1d168\""
Sep 10 13:43:41.006066 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.006044    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"]
Sep 10 13:43:41.021025 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.021005    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"]
Sep 10 13:43:41.024226 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.024206    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:43:41.026575 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.026560    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-f1d168-dockercfg-wprh4\""
Sep 10 13:43:41.035246 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.035223    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"]
Sep 10 13:43:41.099825 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.099798    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq9g8\" (UniqueName: \"kubernetes.io/projected/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-kube-api-access-vq9g8\") pod \"payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg\" (UID: \"60a72794-0b71-4ed3-b35a-cfecf2e07a3f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:43:41.099976 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.099908    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg\" (UID: \"60a72794-0b71-4ed3-b35a-cfecf2e07a3f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:43:41.100026 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.099972    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tvnb\" (UniqueName: \"kubernetes.io/projected/33d5347d-2814-48e4-8fa1-0a93113e514b-kube-api-access-5tvnb\") pod \"payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9\" (UID: \"33d5347d-2814-48e4-8fa1-0a93113e514b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:43:41.100071 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.100034    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d5347d-2814-48e4-8fa1-0a93113e514b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9\" (UID: \"33d5347d-2814-48e4-8fa1-0a93113e514b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:43:41.201346 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.201314    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg\" (UID: \"60a72794-0b71-4ed3-b35a-cfecf2e07a3f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:43:41.201518 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.201362    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvnb\" (UniqueName: \"kubernetes.io/projected/33d5347d-2814-48e4-8fa1-0a93113e514b-kube-api-access-5tvnb\") pod \"payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9\" (UID: \"33d5347d-2814-48e4-8fa1-0a93113e514b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:43:41.201518 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.201409    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d5347d-2814-48e4-8fa1-0a93113e514b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9\" (UID: \"33d5347d-2814-48e4-8fa1-0a93113e514b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:43:41.201518 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.201462    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq9g8\" (UniqueName: \"kubernetes.io/projected/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-kube-api-access-vq9g8\") pod \"payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg\" (UID: \"60a72794-0b71-4ed3-b35a-cfecf2e07a3f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:43:41.202099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.202076    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d5347d-2814-48e4-8fa1-0a93113e514b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9\" (UID: \"33d5347d-2814-48e4-8fa1-0a93113e514b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:43:41.202171 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.202129    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg\" (UID: \"60a72794-0b71-4ed3-b35a-cfecf2e07a3f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:43:41.213897 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.211094    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq9g8\" (UniqueName: \"kubernetes.io/projected/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-kube-api-access-vq9g8\") pod \"payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg\" (UID: \"60a72794-0b71-4ed3-b35a-cfecf2e07a3f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:43:41.214619 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.214599    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tvnb\" (UniqueName: \"kubernetes.io/projected/33d5347d-2814-48e4-8fa1-0a93113e514b-kube-api-access-5tvnb\") pod \"payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9\" (UID: \"33d5347d-2814-48e4-8fa1-0a93113e514b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:43:41.305761 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.305690    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:43:41.404512 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.404475    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:43:41.539277 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.539247    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7853-66e934" event={"ID":"0e5dbd52-786e-4de0-a9be-0a8715e08185","Type":"ContainerStarted","Data":"7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4"}
Sep 10 13:43:41.564874 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.564787    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7853-66e934" podStartSLOduration=1.362997343 podStartE2EDuration="3.56477245s" totalImagesPullingTime="2.201775107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:41.56182197 +0000 UTC m=+780.468379999" watchObservedRunningTime="2026-09-10 13:43:41.56477245 +0000 UTC m=+780.471330479"
Sep 10 13:43:41.605980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.605957    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"]
Sep 10 13:43:41.607961 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:43:41.607935    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d5347d_2814_48e4_8fa1_0a93113e514b.slice/crio-969e361790727eea50679b05beda1971a3bb1a36063faa736a48f6e0330415ed WatchSource:0}: Error finding container 969e361790727eea50679b05beda1971a3bb1a36063faa736a48f6e0330415ed: Status 404 returned error can't find the container with id 969e361790727eea50679b05beda1971a3bb1a36063faa736a48f6e0330415ed
Sep 10 13:43:41.682948 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:41.682799    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"]
Sep 10 13:43:41.685357 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:43:41.685327    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a72794_0b71_4ed3_b35a_cfecf2e07a3f.slice/crio-80fc909cc0404600037108dd8db467710732096f9d6aae7cd10e4d3d14b873f8 WatchSource:0}: Error finding container 80fc909cc0404600037108dd8db467710732096f9d6aae7cd10e4d3d14b873f8: Status 404 returned error can't find the container with id 80fc909cc0404600037108dd8db467710732096f9d6aae7cd10e4d3d14b873f8
Sep 10 13:43:42.260534 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:42.260499    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7853-66e934"]
Sep 10 13:43:42.262344 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:42.262316    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7853-66e934"]
Sep 10 13:43:42.544656 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:42.544617    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg" event={"ID":"60a72794-0b71-4ed3-b35a-cfecf2e07a3f","Type":"ContainerStarted","Data":"f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc"}
Sep 10 13:43:42.544841 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:42.544672    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg" event={"ID":"60a72794-0b71-4ed3-b35a-cfecf2e07a3f","Type":"ContainerStarted","Data":"80fc909cc0404600037108dd8db467710732096f9d6aae7cd10e4d3d14b873f8"}
Sep 10 13:43:42.544841 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:42.544733    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:43:42.546231 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:42.546208    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9" event={"ID":"33d5347d-2814-48e4-8fa1-0a93113e514b","Type":"ContainerStarted","Data":"479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134"}
Sep 10 13:43:42.546341 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:42.546248    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9" event={"ID":"33d5347d-2814-48e4-8fa1-0a93113e514b","Type":"ContainerStarted","Data":"969e361790727eea50679b05beda1971a3bb1a36063faa736a48f6e0330415ed"}
Sep 10 13:43:42.546341 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:42.546332    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:43:42.546428 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:43:42.546410    2549 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7853-66e934" container="test-curl-7853-66e934"
Sep 10 13:43:42.569928 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:42.569879    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg" podStartSLOduration=1.569846129 podStartE2EDuration="1.569846129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:42.568159469 +0000 UTC m=+781.474717511" watchObservedRunningTime="2026-09-10 13:43:42.569846129 +0000 UTC m=+781.476404162"
Sep 10 13:43:42.594976 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:42.594928    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9" podStartSLOduration=2.594914337 podStartE2EDuration="2.594914337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:42.592956044 +0000 UTC m=+781.499514071" watchObservedRunningTime="2026-09-10 13:43:42.594914337 +0000 UTC m=+781.501472368"
Sep 10 13:43:43.550565 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:43.550523    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7853-66e934" podUID="0e5dbd52-786e-4de0-a9be-0a8715e08185" containerName="test-curl-7853-66e934" containerID="cri-o://7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4" gracePeriod=2
Sep 10 13:43:43.556122 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:43:43.556091    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7853-66e934\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="0e5dbd52-786e-4de0-a9be-0a8715e08185" pod="opendatahub/test-curl-7853-66e934"
Sep 10 13:43:45.729465 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:45.729441    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7853-66e934"
Sep 10 13:43:45.840583 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:45.840506    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdsfk\" (UniqueName: \"kubernetes.io/projected/0e5dbd52-786e-4de0-a9be-0a8715e08185-kube-api-access-sdsfk\") pod \"0e5dbd52-786e-4de0-a9be-0a8715e08185\" (UID: \"0e5dbd52-786e-4de0-a9be-0a8715e08185\") "
Sep 10 13:43:45.842403 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:45.842385    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e5dbd52-786e-4de0-a9be-0a8715e08185-kube-api-access-sdsfk" (OuterVolumeSpecName: "kube-api-access-sdsfk") pod "0e5dbd52-786e-4de0-a9be-0a8715e08185" (UID: "0e5dbd52-786e-4de0-a9be-0a8715e08185"). InnerVolumeSpecName "kube-api-access-sdsfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:43:45.941898 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:45.941855    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdsfk\" (UniqueName: \"kubernetes.io/projected/0e5dbd52-786e-4de0-a9be-0a8715e08185-kube-api-access-sdsfk\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:43:46.564045 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:46.564005    2549 generic.go:356] "Generic (PLEG): container finished" podID="0e5dbd52-786e-4de0-a9be-0a8715e08185" containerID="7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4" exitCode=137
Sep 10 13:43:46.564234 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:46.564050    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7853-66e934"
Sep 10 13:43:46.564234 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:46.564107    2549 scope.go:117] "RemoveContainer" containerID="7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4"
Sep 10 13:43:46.571995 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:46.571974    2549 scope.go:117] "RemoveContainer" containerID="7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4"
Sep 10 13:43:46.572222 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:43:46.572205    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4\": container with ID starting with 7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4 not found: ID does not exist" containerID="7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4"
Sep 10 13:43:46.572285 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:46.572236    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4"} err="failed to get container status \"7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4\": rpc error: code = NotFound desc = could not find container \"7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4\": container with ID starting with 7d943966b88db1de0db91e5fa14cfd085c94c8d5ef2b64210a26aa1277cc39e4 not found: ID does not exist"
Sep 10 13:43:46.573960 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:43:46.573938    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7853-66e934\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="0e5dbd52-786e-4de0-a9be-0a8715e08185" pod="opendatahub/test-curl-7853-66e934"
Sep 10 13:43:47.674816 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:47.674781    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e5dbd52-786e-4de0-a9be-0a8715e08185" path="/var/lib/kubelet/pods/0e5dbd52-786e-4de0-a9be-0a8715e08185/volumes"
Sep 10 13:43:51.259784 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.259748    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"]
Sep 10 13:43:51.260153 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.260076    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e5dbd52-786e-4de0-a9be-0a8715e08185" containerName="test-curl-7853-66e934"
Sep 10 13:43:51.260153 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.260087    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e5dbd52-786e-4de0-a9be-0a8715e08185" containerName="test-curl-7853-66e934"
Sep 10 13:43:51.260153 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.260131    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e5dbd52-786e-4de0-a9be-0a8715e08185" containerName="test-curl-7853-66e934"
Sep 10 13:43:51.264699 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.264683    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:43:51.267318 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.267297    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e9a01385\""
Sep 10 13:43:51.277104 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.277081    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"]
Sep 10 13:43:51.330634 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.330606    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"]
Sep 10 13:43:51.333931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.333916    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:43:51.336496 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.336474    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e9a01385-dockercfg-g79cl\""
Sep 10 13:43:51.347721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.347699    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"]
Sep 10 13:43:51.388469 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.388445    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rwgf\" (UniqueName: \"kubernetes.io/projected/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-kube-api-access-6rwgf\") pod \"payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx\" (UID: \"eb8db34c-93a2-4b03-b2ae-ad562654b9c6\") " pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:43:51.388603 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.388569    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-plugins-config-volume\") pod \"payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx\" (UID: \"eb8db34c-93a2-4b03-b2ae-ad562654b9c6\") " pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:43:51.489776 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.489745    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rwgf\" (UniqueName: \"kubernetes.io/projected/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-kube-api-access-6rwgf\") pod \"payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx\" (UID: \"eb8db34c-93a2-4b03-b2ae-ad562654b9c6\") " pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:43:51.489776 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.489780    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56rzs\" (UniqueName: \"kubernetes.io/projected/ec235a44-8b55-419b-b768-c05ea623c01b-kube-api-access-56rzs\") pod \"payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5\" (UID: \"ec235a44-8b55-419b-b768-c05ea623c01b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:43:51.489999 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.489912    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec235a44-8b55-419b-b768-c05ea623c01b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5\" (UID: \"ec235a44-8b55-419b-b768-c05ea623c01b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:43:51.489999 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.489940    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-plugins-config-volume\") pod \"payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx\" (UID: \"eb8db34c-93a2-4b03-b2ae-ad562654b9c6\") " pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:43:51.490534 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.490516    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-plugins-config-volume\") pod \"payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx\" (UID: \"eb8db34c-93a2-4b03-b2ae-ad562654b9c6\") " pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:43:51.500875 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.500843    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rwgf\" (UniqueName: \"kubernetes.io/projected/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-kube-api-access-6rwgf\") pod \"payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx\" (UID: \"eb8db34c-93a2-4b03-b2ae-ad562654b9c6\") " pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:43:51.574923 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.574831    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:43:51.590368 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.590337    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56rzs\" (UniqueName: \"kubernetes.io/projected/ec235a44-8b55-419b-b768-c05ea623c01b-kube-api-access-56rzs\") pod \"payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5\" (UID: \"ec235a44-8b55-419b-b768-c05ea623c01b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:43:51.590489 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.590448    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec235a44-8b55-419b-b768-c05ea623c01b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5\" (UID: \"ec235a44-8b55-419b-b768-c05ea623c01b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:43:51.591408 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.591384    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec235a44-8b55-419b-b768-c05ea623c01b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5\" (UID: \"ec235a44-8b55-419b-b768-c05ea623c01b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:43:51.604777 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.604749    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56rzs\" (UniqueName: \"kubernetes.io/projected/ec235a44-8b55-419b-b768-c05ea623c01b-kube-api-access-56rzs\") pod \"payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5\" (UID: \"ec235a44-8b55-419b-b768-c05ea623c01b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:43:51.642639 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.642610    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:43:51.846387 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.846235    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"]
Sep 10 13:43:51.849088 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:43:51.849055    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8db34c_93a2_4b03_b2ae_ad562654b9c6.slice/crio-3a02d9847e68fe7d75978ca1c657455d5d2cf5a7d54d077324756154dc2e467e WatchSource:0}: Error finding container 3a02d9847e68fe7d75978ca1c657455d5d2cf5a7d54d077324756154dc2e467e: Status 404 returned error can't find the container with id 3a02d9847e68fe7d75978ca1c657455d5d2cf5a7d54d077324756154dc2e467e
Sep 10 13:43:51.919218 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:51.919146    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"]
Sep 10 13:43:51.921639 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:43:51.921617    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec235a44_8b55_419b_b768_c05ea623c01b.slice/crio-77900cb50921d0bc152b548194248679be584a1ec24a833a83fe3cf79f50b338 WatchSource:0}: Error finding container 77900cb50921d0bc152b548194248679be584a1ec24a833a83fe3cf79f50b338: Status 404 returned error can't find the container with id 77900cb50921d0bc152b548194248679be584a1ec24a833a83fe3cf79f50b338
Sep 10 13:43:52.592894 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:52.592841    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx" event={"ID":"eb8db34c-93a2-4b03-b2ae-ad562654b9c6","Type":"ContainerStarted","Data":"0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b"}
Sep 10 13:43:52.593325 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:52.593018    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:43:52.593325 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:52.593059    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx" event={"ID":"eb8db34c-93a2-4b03-b2ae-ad562654b9c6","Type":"ContainerStarted","Data":"3a02d9847e68fe7d75978ca1c657455d5d2cf5a7d54d077324756154dc2e467e"}
Sep 10 13:43:52.594425 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:52.594403    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5" event={"ID":"ec235a44-8b55-419b-b768-c05ea623c01b","Type":"ContainerStarted","Data":"5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f"}
Sep 10 13:43:52.594533 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:52.594443    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5" event={"ID":"ec235a44-8b55-419b-b768-c05ea623c01b","Type":"ContainerStarted","Data":"77900cb50921d0bc152b548194248679be584a1ec24a833a83fe3cf79f50b338"}
Sep 10 13:43:52.594533 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:52.594480    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:43:52.618144 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:52.618103    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx" podStartSLOduration=1.618090283 podStartE2EDuration="1.618090283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:52.616118657 +0000 UTC m=+791.522676687" watchObservedRunningTime="2026-09-10 13:43:52.618090283 +0000 UTC m=+791.524648312"
Sep 10 13:43:52.636551 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:52.636500    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5" podStartSLOduration=1.63648286 podStartE2EDuration="1.63648286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:52.636024691 +0000 UTC m=+791.542582721" watchObservedRunningTime="2026-09-10 13:43:52.63648286 +0000 UTC m=+791.543040891"
Sep 10 13:43:53.552858 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:53.552829    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:43:53.553026 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:43:53.552918    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:44:03.602824 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:03.602795    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:44:03.603253 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:03.603104    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:44:04.933327 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:04.933293    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"]
Sep 10 13:44:04.936906 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:04.936885    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:44:04.940141 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:04.940123    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-e6d037-dockercfg-fb52p\""
Sep 10 13:44:04.940579 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:04.940563    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e6d037\""
Sep 10 13:44:04.951150 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:04.951133    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"]
Sep 10 13:44:05.000929 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.000902    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grnkv\" (UniqueName: \"kubernetes.io/projected/f116b39c-ce19-4577-b14c-36b4ce47a41b-kube-api-access-grnkv\") pod \"payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6\" (UID: \"f116b39c-ce19-4577-b14c-36b4ce47a41b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:44:05.001054 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.000948    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f116b39c-ce19-4577-b14c-36b4ce47a41b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6\" (UID: \"f116b39c-ce19-4577-b14c-36b4ce47a41b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:44:05.021760 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.021737    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"]
Sep 10 13:44:05.024962 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.024944    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:44:05.042355 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.042337    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"]
Sep 10 13:44:05.101968 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.101935    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce6b50d9-3848-4c8e-aafc-099df8986d82-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k\" (UID: \"ce6b50d9-3848-4c8e-aafc-099df8986d82\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:44:05.102096 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.101973    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjzb8\" (UniqueName: \"kubernetes.io/projected/ce6b50d9-3848-4c8e-aafc-099df8986d82-kube-api-access-mjzb8\") pod \"payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k\" (UID: \"ce6b50d9-3848-4c8e-aafc-099df8986d82\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:44:05.102096 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.102006    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grnkv\" (UniqueName: \"kubernetes.io/projected/f116b39c-ce19-4577-b14c-36b4ce47a41b-kube-api-access-grnkv\") pod \"payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6\" (UID: \"f116b39c-ce19-4577-b14c-36b4ce47a41b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:44:05.102096 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.102071    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f116b39c-ce19-4577-b14c-36b4ce47a41b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6\" (UID: \"f116b39c-ce19-4577-b14c-36b4ce47a41b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:44:05.102762 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.102744    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f116b39c-ce19-4577-b14c-36b4ce47a41b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6\" (UID: \"f116b39c-ce19-4577-b14c-36b4ce47a41b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:44:05.112126 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.112100    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grnkv\" (UniqueName: \"kubernetes.io/projected/f116b39c-ce19-4577-b14c-36b4ce47a41b-kube-api-access-grnkv\") pod \"payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6\" (UID: \"f116b39c-ce19-4577-b14c-36b4ce47a41b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:44:05.202727 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.202668    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce6b50d9-3848-4c8e-aafc-099df8986d82-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k\" (UID: \"ce6b50d9-3848-4c8e-aafc-099df8986d82\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:44:05.202727 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.202702    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjzb8\" (UniqueName: \"kubernetes.io/projected/ce6b50d9-3848-4c8e-aafc-099df8986d82-kube-api-access-mjzb8\") pod \"payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k\" (UID: \"ce6b50d9-3848-4c8e-aafc-099df8986d82\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:44:05.203321 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.203303    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce6b50d9-3848-4c8e-aafc-099df8986d82-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k\" (UID: \"ce6b50d9-3848-4c8e-aafc-099df8986d82\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:44:05.221287 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.221262    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjzb8\" (UniqueName: \"kubernetes.io/projected/ce6b50d9-3848-4c8e-aafc-099df8986d82-kube-api-access-mjzb8\") pod \"payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k\" (UID: \"ce6b50d9-3848-4c8e-aafc-099df8986d82\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:44:05.246987 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.246963    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:44:05.334955 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.334924    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:44:05.539687 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.539610    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"]
Sep 10 13:44:05.542879 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:44:05.542839    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf116b39c_ce19_4577_b14c_36b4ce47a41b.slice/crio-072c644f55b1dbbafe7651a07bae2f59fe9ca20be83f5b0d06f8caad0b9c8276 WatchSource:0}: Error finding container 072c644f55b1dbbafe7651a07bae2f59fe9ca20be83f5b0d06f8caad0b9c8276: Status 404 returned error can't find the container with id 072c644f55b1dbbafe7651a07bae2f59fe9ca20be83f5b0d06f8caad0b9c8276
Sep 10 13:44:05.638599 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.638568    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"]
Sep 10 13:44:05.639885 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:44:05.639850    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6b50d9_3848_4c8e_aafc_099df8986d82.slice/crio-f312833d5470d49c7ed889d9482decec1b26475d6a80bbd96268926e319c7833 WatchSource:0}: Error finding container f312833d5470d49c7ed889d9482decec1b26475d6a80bbd96268926e319c7833: Status 404 returned error can't find the container with id f312833d5470d49c7ed889d9482decec1b26475d6a80bbd96268926e319c7833
Sep 10 13:44:05.652205 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.652084    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6" event={"ID":"f116b39c-ce19-4577-b14c-36b4ce47a41b","Type":"ContainerStarted","Data":"fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e"}
Sep 10 13:44:05.652205 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.652144    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6" event={"ID":"f116b39c-ce19-4577-b14c-36b4ce47a41b","Type":"ContainerStarted","Data":"072c644f55b1dbbafe7651a07bae2f59fe9ca20be83f5b0d06f8caad0b9c8276"}
Sep 10 13:44:05.652205 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.652172    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:44:05.653439 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.653416    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k" event={"ID":"ce6b50d9-3848-4c8e-aafc-099df8986d82","Type":"ContainerStarted","Data":"f312833d5470d49c7ed889d9482decec1b26475d6a80bbd96268926e319c7833"}
Sep 10 13:44:05.675244 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:05.675196    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6" podStartSLOduration=1.6751804510000001 podStartE2EDuration="1.675180451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:05.673832855 +0000 UTC m=+804.580390906" watchObservedRunningTime="2026-09-10 13:44:05.675180451 +0000 UTC m=+804.581738481"
Sep 10 13:44:06.659841 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:06.659800    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k" event={"ID":"ce6b50d9-3848-4c8e-aafc-099df8986d82","Type":"ContainerStarted","Data":"228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550"}
Sep 10 13:44:06.660299 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:06.659999    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:44:06.685974 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:06.685928    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k" podStartSLOduration=2.685914848 podStartE2EDuration="2.685914848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:06.683891077 +0000 UTC m=+805.590449110" watchObservedRunningTime="2026-09-10 13:44:06.685914848 +0000 UTC m=+805.592472879"
Sep 10 13:44:15.062895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.062848    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"]
Sep 10 13:44:15.063431 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.063150    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx" podUID="eb8db34c-93a2-4b03-b2ae-ad562654b9c6" containerName="payload-processing" containerID="cri-o://0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b" gracePeriod=30
Sep 10 13:44:15.075793 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.075769    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"]
Sep 10 13:44:15.076030 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.076009    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5" podUID="ec235a44-8b55-419b-b768-c05ea623c01b" containerName="payload-pre-processing" containerID="cri-o://5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f" gracePeriod=30
Sep 10 13:44:15.310683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.310657    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:44:15.336032 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.336012    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:44:15.391425 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.391398    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rwgf\" (UniqueName: \"kubernetes.io/projected/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-kube-api-access-6rwgf\") pod \"eb8db34c-93a2-4b03-b2ae-ad562654b9c6\" (UID: \"eb8db34c-93a2-4b03-b2ae-ad562654b9c6\") "
Sep 10 13:44:15.391550 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.391483    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec235a44-8b55-419b-b768-c05ea623c01b-plugins-config-volume\") pod \"ec235a44-8b55-419b-b768-c05ea623c01b\" (UID: \"ec235a44-8b55-419b-b768-c05ea623c01b\") "
Sep 10 13:44:15.391550 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.391511    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-plugins-config-volume\") pod \"eb8db34c-93a2-4b03-b2ae-ad562654b9c6\" (UID: \"eb8db34c-93a2-4b03-b2ae-ad562654b9c6\") "
Sep 10 13:44:15.391679 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.391566    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56rzs\" (UniqueName: \"kubernetes.io/projected/ec235a44-8b55-419b-b768-c05ea623c01b-kube-api-access-56rzs\") pod \"ec235a44-8b55-419b-b768-c05ea623c01b\" (UID: \"ec235a44-8b55-419b-b768-c05ea623c01b\") "
Sep 10 13:44:15.392004 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.391957    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec235a44-8b55-419b-b768-c05ea623c01b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec235a44-8b55-419b-b768-c05ea623c01b" (UID: "ec235a44-8b55-419b-b768-c05ea623c01b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:44:15.392116 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.392083    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb8db34c-93a2-4b03-b2ae-ad562654b9c6" (UID: "eb8db34c-93a2-4b03-b2ae-ad562654b9c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:44:15.394221 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.394200    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec235a44-8b55-419b-b768-c05ea623c01b-kube-api-access-56rzs" (OuterVolumeSpecName: "kube-api-access-56rzs") pod "ec235a44-8b55-419b-b768-c05ea623c01b" (UID: "ec235a44-8b55-419b-b768-c05ea623c01b"). InnerVolumeSpecName "kube-api-access-56rzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:44:15.394341 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.394323    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-kube-api-access-6rwgf" (OuterVolumeSpecName: "kube-api-access-6rwgf") pod "eb8db34c-93a2-4b03-b2ae-ad562654b9c6" (UID: "eb8db34c-93a2-4b03-b2ae-ad562654b9c6"). InnerVolumeSpecName "kube-api-access-6rwgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:44:15.492409 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.492380    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56rzs\" (UniqueName: \"kubernetes.io/projected/ec235a44-8b55-419b-b768-c05ea623c01b-kube-api-access-56rzs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:44:15.492409 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.492409    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rwgf\" (UniqueName: \"kubernetes.io/projected/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-kube-api-access-6rwgf\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:44:15.492585 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.492421    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec235a44-8b55-419b-b768-c05ea623c01b-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:44:15.492585 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.492430    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb8db34c-93a2-4b03-b2ae-ad562654b9c6-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:44:15.697052 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.697022    2549 generic.go:356] "Generic (PLEG): container finished" podID="eb8db34c-93a2-4b03-b2ae-ad562654b9c6" containerID="0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b" exitCode=0
Sep 10 13:44:15.697159 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.697061    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"
Sep 10 13:44:15.697159 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.697098    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx" event={"ID":"eb8db34c-93a2-4b03-b2ae-ad562654b9c6","Type":"ContainerDied","Data":"0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b"}
Sep 10 13:44:15.697159 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.697146    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx" event={"ID":"eb8db34c-93a2-4b03-b2ae-ad562654b9c6","Type":"ContainerDied","Data":"3a02d9847e68fe7d75978ca1c657455d5d2cf5a7d54d077324756154dc2e467e"}
Sep 10 13:44:15.697311 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.697169    2549 scope.go:117] "RemoveContainer" containerID="0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b"
Sep 10 13:44:15.698484 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.698463    2549 generic.go:356] "Generic (PLEG): container finished" podID="ec235a44-8b55-419b-b768-c05ea623c01b" containerID="5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f" exitCode=0
Sep 10 13:44:15.698599 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.698503    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"
Sep 10 13:44:15.698599 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.698528    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5" event={"ID":"ec235a44-8b55-419b-b768-c05ea623c01b","Type":"ContainerDied","Data":"5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f"}
Sep 10 13:44:15.698599 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.698579    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5" event={"ID":"ec235a44-8b55-419b-b768-c05ea623c01b","Type":"ContainerDied","Data":"77900cb50921d0bc152b548194248679be584a1ec24a833a83fe3cf79f50b338"}
Sep 10 13:44:15.705441 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.705426    2549 scope.go:117] "RemoveContainer" containerID="0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b"
Sep 10 13:44:15.705708 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:44:15.705691    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b\": container with ID starting with 0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b not found: ID does not exist" containerID="0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b"
Sep 10 13:44:15.705753 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.705715    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b"} err="failed to get container status \"0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b\": rpc error: code = NotFound desc = could not find container \"0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b\": container with ID starting with 0e7b2d20f59aa98205703559e23ad945e876257539db7517c035213feb5e6b3b not found: ID does not exist"
Sep 10 13:44:15.705753 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.705732    2549 scope.go:117] "RemoveContainer" containerID="5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f"
Sep 10 13:44:15.712672 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.712652    2549 scope.go:117] "RemoveContainer" containerID="5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f"
Sep 10 13:44:15.712913 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:44:15.712897    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f\": container with ID starting with 5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f not found: ID does not exist" containerID="5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f"
Sep 10 13:44:15.712979 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.712918    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f"} err="failed to get container status \"5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f\": rpc error: code = NotFound desc = could not find container \"5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f\": container with ID starting with 5f6f6d9c538177b6412e1296c91a16154e1307a90bc4fa95b96079b58ca0592f not found: ID does not exist"
Sep 10 13:44:15.804436 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.804402    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"]
Sep 10 13:44:15.808442 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.808419    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9a01385-587b68f5db-cx7q5"]
Sep 10 13:44:15.855952 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.855927    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"]
Sep 10 13:44:15.862785 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:15.862762    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9a01385-9cb679b79-szqcx"]
Sep 10 13:44:16.662295 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:16.662268    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:44:17.665877 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:17.665845    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:44:17.673609 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:17.673583    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb8db34c-93a2-4b03-b2ae-ad562654b9c6" path="/var/lib/kubelet/pods/eb8db34c-93a2-4b03-b2ae-ad562654b9c6/volumes"
Sep 10 13:44:17.673964 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:17.673950    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec235a44-8b55-419b-b768-c05ea623c01b" path="/var/lib/kubelet/pods/ec235a44-8b55-419b-b768-c05ea623c01b/volumes"
Sep 10 13:44:27.034791 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.034761    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"]
Sep 10 13:44:27.035185 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.035070    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb8db34c-93a2-4b03-b2ae-ad562654b9c6" containerName="payload-processing"
Sep 10 13:44:27.035185 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.035081    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb8db34c-93a2-4b03-b2ae-ad562654b9c6" containerName="payload-processing"
Sep 10 13:44:27.035185 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.035089    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec235a44-8b55-419b-b768-c05ea623c01b" containerName="payload-pre-processing"
Sep 10 13:44:27.035185 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.035094    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec235a44-8b55-419b-b768-c05ea623c01b" containerName="payload-pre-processing"
Sep 10 13:44:27.035185 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.035141    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb8db34c-93a2-4b03-b2ae-ad562654b9c6" containerName="payload-processing"
Sep 10 13:44:27.035185 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.035148    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec235a44-8b55-419b-b768-c05ea623c01b" containerName="payload-pre-processing"
Sep 10 13:44:27.039198 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.039177    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:27.044370 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.044351    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b1d82512\""
Sep 10 13:44:27.056447 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.056428    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"]
Sep 10 13:44:27.060084 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.060070    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:27.075618 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.075591    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"]
Sep 10 13:44:27.078615 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.078596    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.090705 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.090679    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"]
Sep 10 13:44:27.090950 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.090932    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tznxz\" (UniqueName: \"kubernetes.io/projected/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-kube-api-access-tznxz\") pod \"payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf\" (UID: \"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b\") " pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:27.090996 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.090967    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf\" (UID: \"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b\") " pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:27.093091 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.093074    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-3696db05-kserve-self-signed-certs\""
Sep 10 13:44:27.131993 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.131970    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"]
Sep 10 13:44:27.134772 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.134749    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"]
Sep 10 13:44:27.192037 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192014    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/793134fa-51be-4587-a225-c444c8990150-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj\" (UID: \"793134fa-51be-4587-a225-c444c8990150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:27.192175 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192043    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tls-certs\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.192175 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192061    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-home\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.192175 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192090    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2rp4\" (UniqueName: \"kubernetes.io/projected/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-kube-api-access-x2rp4\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.192175 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192157    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tmp-dir\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.192347 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192176    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74vw6\" (UniqueName: \"kubernetes.io/projected/793134fa-51be-4587-a225-c444c8990150-kube-api-access-74vw6\") pod \"payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj\" (UID: \"793134fa-51be-4587-a225-c444c8990150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:27.192347 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192197    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tznxz\" (UniqueName: \"kubernetes.io/projected/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-kube-api-access-tznxz\") pod \"payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf\" (UID: \"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b\") " pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:27.192347 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192227    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf\" (UID: \"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b\") " pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:27.192347 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192252    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-model-cache\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.192525 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192352    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-dshm\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.192853 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.192834    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf\" (UID: \"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b\") " pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:27.208920 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.208898    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tznxz\" (UniqueName: \"kubernetes.io/projected/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-kube-api-access-tznxz\") pod \"payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf\" (UID: \"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b\") " pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:27.293488 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.293411    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/793134fa-51be-4587-a225-c444c8990150-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj\" (UID: \"793134fa-51be-4587-a225-c444c8990150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:27.293488 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.293451    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tls-certs\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.293488 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.293475    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-home\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.293731 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.293503    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2rp4\" (UniqueName: \"kubernetes.io/projected/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-kube-api-access-x2rp4\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.293731 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.293541    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tmp-dir\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.293731 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.293565    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74vw6\" (UniqueName: \"kubernetes.io/projected/793134fa-51be-4587-a225-c444c8990150-kube-api-access-74vw6\") pod \"payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj\" (UID: \"793134fa-51be-4587-a225-c444c8990150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:27.293731 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.293622    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-model-cache\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.293731 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.293656    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-dshm\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.293970 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.293908    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-home\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.293970 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.293930    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tmp-dir\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.294177 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.294158    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-model-cache\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.294287 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.294262    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/793134fa-51be-4587-a225-c444c8990150-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj\" (UID: \"793134fa-51be-4587-a225-c444c8990150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:27.295785 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.295761    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-dshm\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.296046 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.296031    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tls-certs\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.307823 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.307801    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2rp4\" (UniqueName: \"kubernetes.io/projected/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-kube-api-access-x2rp4\") pod \"e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") " pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.307939 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.307879    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74vw6\" (UniqueName: \"kubernetes.io/projected/793134fa-51be-4587-a225-c444c8990150-kube-api-access-74vw6\") pod \"payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj\" (UID: \"793134fa-51be-4587-a225-c444c8990150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:27.348714 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.348686    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:27.368966 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.368947    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:27.387986 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.387955    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:27.641675 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:44:27.641645    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d8f113_dbb9_4c6c_95c3_9310ea3ae66b.slice/crio-f976a87a7b1cb111afdf20ad379beeaffd700eada455e5da50907e65b8684bd2 WatchSource:0}: Error finding container f976a87a7b1cb111afdf20ad379beeaffd700eada455e5da50907e65b8684bd2: Status 404 returned error can't find the container with id f976a87a7b1cb111afdf20ad379beeaffd700eada455e5da50907e65b8684bd2
Sep 10 13:44:27.645788 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.645760    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"]
Sep 10 13:44:27.693187 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.693165    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"]
Sep 10 13:44:27.694649 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:44:27.694624    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793134fa_51be_4587_a225_c444c8990150.slice/crio-f14055c1536f9bbce72f876d585b3c091c91b7d9cb59cd2e2c93cd4e27cb80fa WatchSource:0}: Error finding container f14055c1536f9bbce72f876d585b3c091c91b7d9cb59cd2e2c93cd4e27cb80fa: Status 404 returned error can't find the container with id f14055c1536f9bbce72f876d585b3c091c91b7d9cb59cd2e2c93cd4e27cb80fa
Sep 10 13:44:27.725221 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.725150    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"]
Sep 10 13:44:27.728269 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:44:27.728246    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaab3722f_ed15_4e0d_b15b_33001ad1a5d9.slice/crio-a5e63717be52cbf653ab14723e3f2c6ecafca6c7cd73b3fbfc07ea0fccbcb9d4 WatchSource:0}: Error finding container a5e63717be52cbf653ab14723e3f2c6ecafca6c7cd73b3fbfc07ea0fccbcb9d4: Status 404 returned error can't find the container with id a5e63717be52cbf653ab14723e3f2c6ecafca6c7cd73b3fbfc07ea0fccbcb9d4
Sep 10 13:44:27.749323 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.749187    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf" event={"ID":"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b","Type":"ContainerStarted","Data":"c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca"}
Sep 10 13:44:27.749323 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.749241    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf" event={"ID":"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b","Type":"ContainerStarted","Data":"f976a87a7b1cb111afdf20ad379beeaffd700eada455e5da50907e65b8684bd2"}
Sep 10 13:44:27.749323 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.749285    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:27.750755 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.750732    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg" event={"ID":"aab3722f-ed15-4e0d-b15b-33001ad1a5d9","Type":"ContainerStarted","Data":"a5e63717be52cbf653ab14723e3f2c6ecafca6c7cd73b3fbfc07ea0fccbcb9d4"}
Sep 10 13:44:27.752241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.752221    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj" event={"ID":"793134fa-51be-4587-a225-c444c8990150","Type":"ContainerStarted","Data":"f14055c1536f9bbce72f876d585b3c091c91b7d9cb59cd2e2c93cd4e27cb80fa"}
Sep 10 13:44:27.810746 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:27.810691    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf" podStartSLOduration=0.810675127 podStartE2EDuration="810.675127ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:27.808540599 +0000 UTC m=+826.715098629" watchObservedRunningTime="2026-09-10 13:44:27.810675127 +0000 UTC m=+826.717233157"
Sep 10 13:44:28.758057 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:28.758023    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg" event={"ID":"aab3722f-ed15-4e0d-b15b-33001ad1a5d9","Type":"ContainerStarted","Data":"a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8"}
Sep 10 13:44:28.759638 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:28.759605    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj" event={"ID":"793134fa-51be-4587-a225-c444c8990150","Type":"ContainerStarted","Data":"49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51"}
Sep 10 13:44:28.759781 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:28.759733    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:28.790438 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:28.790383    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg" podStartSLOduration=1.7903656959999998 podStartE2EDuration="1.790365696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:28.78895631 +0000 UTC m=+827.695514340" watchObservedRunningTime="2026-09-10 13:44:28.790365696 +0000 UTC m=+827.696923726"
Sep 10 13:44:28.880413 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:28.880359    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj" podStartSLOduration=1.880344169 podStartE2EDuration="1.880344169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:28.878310934 +0000 UTC m=+827.784868963" watchObservedRunningTime="2026-09-10 13:44:28.880344169 +0000 UTC m=+827.786902198"
Sep 10 13:44:37.388731 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:37.388693    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:37.389110 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:37.388743    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:37.400932 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:37.400909    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:37.806669 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:37.806641    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:44:38.761764 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:38.761734    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:39.765580 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:39.765554    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:51.009720 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.009683    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"]
Sep 10 13:44:51.010302 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.009900    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf" podUID="94d8f113-dbb9-4c6c-95c3-9310ea3ae66b" containerName="payload-processing" containerID="cri-o://c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca" gracePeriod=30
Sep 10 13:44:51.014597 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.014562    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"]
Sep 10 13:44:51.014755 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.014734    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj" podUID="793134fa-51be-4587-a225-c444c8990150" containerName="payload-pre-processing" containerID="cri-o://49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51" gracePeriod=30
Sep 10 13:44:51.273158 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.273103    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:51.292538 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.292513    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:51.399853 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.399822    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74vw6\" (UniqueName: \"kubernetes.io/projected/793134fa-51be-4587-a225-c444c8990150-kube-api-access-74vw6\") pod \"793134fa-51be-4587-a225-c444c8990150\" (UID: \"793134fa-51be-4587-a225-c444c8990150\") "
Sep 10 13:44:51.400058 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.399920    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-plugins-config-volume\") pod \"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b\" (UID: \"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b\") "
Sep 10 13:44:51.400058 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.399954    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/793134fa-51be-4587-a225-c444c8990150-plugins-config-volume\") pod \"793134fa-51be-4587-a225-c444c8990150\" (UID: \"793134fa-51be-4587-a225-c444c8990150\") "
Sep 10 13:44:51.400058 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.399981    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tznxz\" (UniqueName: \"kubernetes.io/projected/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-kube-api-access-tznxz\") pod \"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b\" (UID: \"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b\") "
Sep 10 13:44:51.400327 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.400300    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/793134fa-51be-4587-a225-c444c8990150-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "793134fa-51be-4587-a225-c444c8990150" (UID: "793134fa-51be-4587-a225-c444c8990150"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:44:51.400409 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.400305    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "94d8f113-dbb9-4c6c-95c3-9310ea3ae66b" (UID: "94d8f113-dbb9-4c6c-95c3-9310ea3ae66b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:44:51.402011 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.401987    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-kube-api-access-tznxz" (OuterVolumeSpecName: "kube-api-access-tznxz") pod "94d8f113-dbb9-4c6c-95c3-9310ea3ae66b" (UID: "94d8f113-dbb9-4c6c-95c3-9310ea3ae66b"). InnerVolumeSpecName "kube-api-access-tznxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:44:51.402099 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.402080    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/793134fa-51be-4587-a225-c444c8990150-kube-api-access-74vw6" (OuterVolumeSpecName: "kube-api-access-74vw6") pod "793134fa-51be-4587-a225-c444c8990150" (UID: "793134fa-51be-4587-a225-c444c8990150"). InnerVolumeSpecName "kube-api-access-74vw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:44:51.501094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.501056    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74vw6\" (UniqueName: \"kubernetes.io/projected/793134fa-51be-4587-a225-c444c8990150-kube-api-access-74vw6\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:44:51.501094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.501086    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:44:51.501094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.501096    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/793134fa-51be-4587-a225-c444c8990150-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:44:51.501309 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.501104    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tznxz\" (UniqueName: \"kubernetes.io/projected/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b-kube-api-access-tznxz\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:44:51.853815 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.853784    2549 generic.go:356] "Generic (PLEG): container finished" podID="793134fa-51be-4587-a225-c444c8990150" containerID="49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51" exitCode=0
Sep 10 13:44:51.854013 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.853848    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"
Sep 10 13:44:51.854013 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.853895    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj" event={"ID":"793134fa-51be-4587-a225-c444c8990150","Type":"ContainerDied","Data":"49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51"}
Sep 10 13:44:51.854013 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.853974    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj" event={"ID":"793134fa-51be-4587-a225-c444c8990150","Type":"ContainerDied","Data":"f14055c1536f9bbce72f876d585b3c091c91b7d9cb59cd2e2c93cd4e27cb80fa"}
Sep 10 13:44:51.854013 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.853997    2549 scope.go:117] "RemoveContainer" containerID="49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51"
Sep 10 13:44:51.855299 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.855279    2549 generic.go:356] "Generic (PLEG): container finished" podID="94d8f113-dbb9-4c6c-95c3-9310ea3ae66b" containerID="c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca" exitCode=0
Sep 10 13:44:51.855403 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.855319    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf" event={"ID":"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b","Type":"ContainerDied","Data":"c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca"}
Sep 10 13:44:51.855403 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.855338    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"
Sep 10 13:44:51.855403 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.855353    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf" event={"ID":"94d8f113-dbb9-4c6c-95c3-9310ea3ae66b","Type":"ContainerDied","Data":"f976a87a7b1cb111afdf20ad379beeaffd700eada455e5da50907e65b8684bd2"}
Sep 10 13:44:51.861971 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.861938    2549 scope.go:117] "RemoveContainer" containerID="49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51"
Sep 10 13:44:51.862194 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:44:51.862176    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51\": container with ID starting with 49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51 not found: ID does not exist" containerID="49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51"
Sep 10 13:44:51.862243 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.862205    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51"} err="failed to get container status \"49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51\": rpc error: code = NotFound desc = could not find container \"49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51\": container with ID starting with 49849ee3f9b3194d568b97e9dd204c7ebdcec528a36319103953b525d7430c51 not found: ID does not exist"
Sep 10 13:44:51.862243 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.862225    2549 scope.go:117] "RemoveContainer" containerID="c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca"
Sep 10 13:44:51.869220 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.869195    2549 scope.go:117] "RemoveContainer" containerID="c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca"
Sep 10 13:44:51.869461 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:44:51.869444    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca\": container with ID starting with c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca not found: ID does not exist" containerID="c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca"
Sep 10 13:44:51.869510 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.869468    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca"} err="failed to get container status \"c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca\": rpc error: code = NotFound desc = could not find container \"c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca\": container with ID starting with c710a6803ab7316adcbd1aac72de80199f5a7c96c4f50caddc906e5a91bd3dca not found: ID does not exist"
Sep 10 13:44:51.884774 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.884755    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"]
Sep 10 13:44:51.890073 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.890055    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b1d82512-665447d6b6-49kpj"]
Sep 10 13:44:51.919773 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.919749    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"]
Sep 10 13:44:51.923333 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:51.923313    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b1d82512-68dd84b8b6-dhbhf"]
Sep 10 13:44:53.673548 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:53.673514    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="793134fa-51be-4587-a225-c444c8990150" path="/var/lib/kubelet/pods/793134fa-51be-4587-a225-c444c8990150/volumes"
Sep 10 13:44:53.673901 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:53.673876    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94d8f113-dbb9-4c6c-95c3-9310ea3ae66b" path="/var/lib/kubelet/pods/94d8f113-dbb9-4c6c-95c3-9310ea3ae66b/volumes"
Sep 10 13:44:57.570932 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.570901    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"]
Sep 10 13:44:57.571283 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.571210    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94d8f113-dbb9-4c6c-95c3-9310ea3ae66b" containerName="payload-processing"
Sep 10 13:44:57.571283 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.571220    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="94d8f113-dbb9-4c6c-95c3-9310ea3ae66b" containerName="payload-processing"
Sep 10 13:44:57.571283 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.571235    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="793134fa-51be-4587-a225-c444c8990150" containerName="payload-pre-processing"
Sep 10 13:44:57.571283 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.571240    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="793134fa-51be-4587-a225-c444c8990150" containerName="payload-pre-processing"
Sep 10 13:44:57.571408 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.571286    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="793134fa-51be-4587-a225-c444c8990150" containerName="payload-pre-processing"
Sep 10 13:44:57.571408 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.571297    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="94d8f113-dbb9-4c6c-95c3-9310ea3ae66b" containerName="payload-processing"
Sep 10 13:44:57.575563 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.575547    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:44:57.577994 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.577968    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0d6a65c4\""
Sep 10 13:44:57.578148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.577996    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0d6a65c4-dockercfg-swplt\""
Sep 10 13:44:57.586360 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.586339    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"]
Sep 10 13:44:57.647763 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.647729    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h67rr\" (UniqueName: \"kubernetes.io/projected/7775ce96-0766-4680-9780-d122fdca46bf-kube-api-access-h67rr\") pod \"payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt\" (UID: \"7775ce96-0766-4680-9780-d122fdca46bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:44:57.647890 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.647788    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7775ce96-0766-4680-9780-d122fdca46bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt\" (UID: \"7775ce96-0766-4680-9780-d122fdca46bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:44:57.673429 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.673406    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"]
Sep 10 13:44:57.676461 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.676446    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:44:57.688114 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.688092    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"]
Sep 10 13:44:57.749152 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.749128    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c9fa08-5fac-4249-baa7-26046e220fb4-plugins-config-volume\") pod \"payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz\" (UID: \"c8c9fa08-5fac-4249-baa7-26046e220fb4\") " pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:44:57.749259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.749186    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h67rr\" (UniqueName: \"kubernetes.io/projected/7775ce96-0766-4680-9780-d122fdca46bf-kube-api-access-h67rr\") pod \"payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt\" (UID: \"7775ce96-0766-4680-9780-d122fdca46bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:44:57.749259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.749222    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzttd\" (UniqueName: \"kubernetes.io/projected/c8c9fa08-5fac-4249-baa7-26046e220fb4-kube-api-access-xzttd\") pod \"payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz\" (UID: \"c8c9fa08-5fac-4249-baa7-26046e220fb4\") " pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:44:57.749259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.749241    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7775ce96-0766-4680-9780-d122fdca46bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt\" (UID: \"7775ce96-0766-4680-9780-d122fdca46bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:44:57.749813 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.749797    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7775ce96-0766-4680-9780-d122fdca46bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt\" (UID: \"7775ce96-0766-4680-9780-d122fdca46bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:44:57.771394 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.771365    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h67rr\" (UniqueName: \"kubernetes.io/projected/7775ce96-0766-4680-9780-d122fdca46bf-kube-api-access-h67rr\") pod \"payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt\" (UID: \"7775ce96-0766-4680-9780-d122fdca46bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:44:57.850349 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.850278    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzttd\" (UniqueName: \"kubernetes.io/projected/c8c9fa08-5fac-4249-baa7-26046e220fb4-kube-api-access-xzttd\") pod \"payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz\" (UID: \"c8c9fa08-5fac-4249-baa7-26046e220fb4\") " pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:44:57.850349 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.850331    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c9fa08-5fac-4249-baa7-26046e220fb4-plugins-config-volume\") pod \"payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz\" (UID: \"c8c9fa08-5fac-4249-baa7-26046e220fb4\") " pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:44:57.850929 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.850912    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c9fa08-5fac-4249-baa7-26046e220fb4-plugins-config-volume\") pod \"payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz\" (UID: \"c8c9fa08-5fac-4249-baa7-26046e220fb4\") " pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:44:57.872763 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.872736    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzttd\" (UniqueName: \"kubernetes.io/projected/c8c9fa08-5fac-4249-baa7-26046e220fb4-kube-api-access-xzttd\") pod \"payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz\" (UID: \"c8c9fa08-5fac-4249-baa7-26046e220fb4\") " pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:44:57.885596 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.885570    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:44:57.986482 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:57.986452    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:44:58.162242 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:58.162221    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"]
Sep 10 13:44:58.163950 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:44:58.163924    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7775ce96_0766_4680_9780_d122fdca46bf.slice/crio-663e67dcf688cbe4e3d86b410569c201fa4aeef1fd1cd101e4d0a9cbe057de9e WatchSource:0}: Error finding container 663e67dcf688cbe4e3d86b410569c201fa4aeef1fd1cd101e4d0a9cbe057de9e: Status 404 returned error can't find the container with id 663e67dcf688cbe4e3d86b410569c201fa4aeef1fd1cd101e4d0a9cbe057de9e
Sep 10 13:44:58.258367 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:58.258338    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"]
Sep 10 13:44:58.260659 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:44:58.260635    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c9fa08_5fac_4249_baa7_26046e220fb4.slice/crio-fc6c8d683b8643c5a789f9486cc4c9d1eab5097c4d3dfa7cc8831a499af43947 WatchSource:0}: Error finding container fc6c8d683b8643c5a789f9486cc4c9d1eab5097c4d3dfa7cc8831a499af43947: Status 404 returned error can't find the container with id fc6c8d683b8643c5a789f9486cc4c9d1eab5097c4d3dfa7cc8831a499af43947
Sep 10 13:44:58.888445 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:58.888415    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt" event={"ID":"7775ce96-0766-4680-9780-d122fdca46bf","Type":"ContainerStarted","Data":"63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58"}
Sep 10 13:44:58.888842 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:58.888473    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt" event={"ID":"7775ce96-0766-4680-9780-d122fdca46bf","Type":"ContainerStarted","Data":"663e67dcf688cbe4e3d86b410569c201fa4aeef1fd1cd101e4d0a9cbe057de9e"}
Sep 10 13:44:58.888842 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:58.888499    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:44:58.890006 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:58.889976    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz" event={"ID":"c8c9fa08-5fac-4249-baa7-26046e220fb4","Type":"ContainerStarted","Data":"e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68"}
Sep 10 13:44:58.890124 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:58.890024    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz" event={"ID":"c8c9fa08-5fac-4249-baa7-26046e220fb4","Type":"ContainerStarted","Data":"fc6c8d683b8643c5a789f9486cc4c9d1eab5097c4d3dfa7cc8831a499af43947"}
Sep 10 13:44:58.890124 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:58.890044    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:44:58.909029 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:58.908989    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt" podStartSLOduration=1.908975227 podStartE2EDuration="1.908975227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:58.907073842 +0000 UTC m=+857.813631872" watchObservedRunningTime="2026-09-10 13:44:58.908975227 +0000 UTC m=+857.815533257"
Sep 10 13:44:58.926653 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:44:58.926608    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz" podStartSLOduration=1.926590075 podStartE2EDuration="1.926590075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:58.925819715 +0000 UTC m=+857.832377745" watchObservedRunningTime="2026-09-10 13:44:58.926590075 +0000 UTC m=+857.833148086"
Sep 10 13:45:00.148164 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.148127    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w"]
Sep 10 13:45:00.151225 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.151207    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817465-57l6v"]
Sep 10 13:45:00.151335 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.151317    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w"
Sep 10 13:45:00.153488 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.153464    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 13:45:00.154185 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.154163    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-57l6v"
Sep 10 13:45:00.160259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.160240    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w"]
Sep 10 13:45:00.165461 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.165438    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817465-57l6v"]
Sep 10 13:45:00.165917 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.165895    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w"
Sep 10 13:45:00.177113 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.177087    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-57l6v"
Sep 10 13:45:00.455539 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.455512    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817465-57l6v"]
Sep 10 13:45:00.458631 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:45:00.458605    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd00cadbf_15b9_4d92_89d6_b96d7137fda1.slice/crio-72e6b3218631886734f54b9a3317f331092346e98f7819b67591264f3d453ffd WatchSource:0}: Error finding container 72e6b3218631886734f54b9a3317f331092346e98f7819b67591264f3d453ffd: Status 404 returned error can't find the container with id 72e6b3218631886734f54b9a3317f331092346e98f7819b67591264f3d453ffd
Sep 10 13:45:00.458908 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.458682    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w"]
Sep 10 13:45:00.460650 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:45:00.460609    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb697720_d6c8_45db_be71_2635a40605ba.slice/crio-ec69e8277ced0e1fbb16480c0755f8fb4c656af0b4ff21a9fd016b13a8b15c84 WatchSource:0}: Error finding container ec69e8277ced0e1fbb16480c0755f8fb4c656af0b4ff21a9fd016b13a8b15c84: Status 404 returned error can't find the container with id ec69e8277ced0e1fbb16480c0755f8fb4c656af0b4ff21a9fd016b13a8b15c84
Sep 10 13:45:00.900223 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.900133    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-57l6v" event={"ID":"d00cadbf-15b9-4d92-89d6-b96d7137fda1","Type":"ContainerStarted","Data":"72e6b3218631886734f54b9a3317f331092346e98f7819b67591264f3d453ffd"}
Sep 10 13:45:00.901345 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:00.901321    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w" event={"ID":"bb697720-d6c8-45db-be71-2635a40605ba","Type":"ContainerStarted","Data":"ec69e8277ced0e1fbb16480c0755f8fb4c656af0b4ff21a9fd016b13a8b15c84"}
Sep 10 13:45:02.911972 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:02.911944    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w" event={"ID":"bb697720-d6c8-45db-be71-2635a40605ba","Type":"ContainerStarted","Data":"647c3ec40f65b9d3dcfd04a9bef9cc9015c3607d1dbfac3de468f47d04af91ed"}
Sep 10 13:45:02.937035 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:02.936994    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w" podStartSLOduration=0.763050393 podStartE2EDuration="2.936978696s" totalImagesPullingTime="2.173928303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:02.936158717 +0000 UTC m=+861.842716747" watchObservedRunningTime="2026-09-10 13:45:02.936978696 +0000 UTC m=+861.843536728"
Sep 10 13:45:03.918379 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:03.918348    2549 generic.go:356] "Generic (PLEG): container finished" podID="d00cadbf-15b9-4d92-89d6-b96d7137fda1" containerID="3baca3fd5bca83a4c2fd83ed4c2cce9ad7e26bfe383764e560b9c5f26811371b" exitCode=0
Sep 10 13:45:03.918760 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:03.918423    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-57l6v" event={"ID":"d00cadbf-15b9-4d92-89d6-b96d7137fda1","Type":"ContainerDied","Data":"3baca3fd5bca83a4c2fd83ed4c2cce9ad7e26bfe383764e560b9c5f26811371b"}
Sep 10 13:45:03.919945 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:03.919923    2549 generic.go:356] "Generic (PLEG): container finished" podID="bb697720-d6c8-45db-be71-2635a40605ba" containerID="647c3ec40f65b9d3dcfd04a9bef9cc9015c3607d1dbfac3de468f47d04af91ed" exitCode=0
Sep 10 13:45:03.920071 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:03.919948    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w" event={"ID":"bb697720-d6c8-45db-be71-2635a40605ba","Type":"ContainerDied","Data":"647c3ec40f65b9d3dcfd04a9bef9cc9015c3607d1dbfac3de468f47d04af91ed"}
Sep 10 13:45:05.054366 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:05.054339    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-57l6v"
Sep 10 13:45:05.083371 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:05.083347    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w"
Sep 10 13:45:05.933318 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:05.933286    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-57l6v"
Sep 10 13:45:05.933537 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:05.933293    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-57l6v" event={"ID":"d00cadbf-15b9-4d92-89d6-b96d7137fda1","Type":"ContainerDied","Data":"72e6b3218631886734f54b9a3317f331092346e98f7819b67591264f3d453ffd"}
Sep 10 13:45:05.933537 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:05.933439    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72e6b3218631886734f54b9a3317f331092346e98f7819b67591264f3d453ffd"
Sep 10 13:45:05.934680 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:05.934663    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w"
Sep 10 13:45:05.934790 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:05.934683    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w" event={"ID":"bb697720-d6c8-45db-be71-2635a40605ba","Type":"ContainerDied","Data":"ec69e8277ced0e1fbb16480c0755f8fb4c656af0b4ff21a9fd016b13a8b15c84"}
Sep 10 13:45:05.934790 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:05.934716    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec69e8277ced0e1fbb16480c0755f8fb4c656af0b4ff21a9fd016b13a8b15c84"
Sep 10 13:45:09.896382 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:09.896351    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:45:09.896782 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:09.896566    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:45:16.290446 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.290405    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"]
Sep 10 13:45:16.290936 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.290904    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb697720-d6c8-45db-be71-2635a40605ba" containerName="cleanup"
Sep 10 13:45:16.290936 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.290922    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb697720-d6c8-45db-be71-2635a40605ba" containerName="cleanup"
Sep 10 13:45:16.291053 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.290951    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00cadbf-15b9-4d92-89d6-b96d7137fda1" containerName="cleanup"
Sep 10 13:45:16.291053 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.290959    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00cadbf-15b9-4d92-89d6-b96d7137fda1" containerName="cleanup"
Sep 10 13:45:16.291053 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.291049    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="d00cadbf-15b9-4d92-89d6-b96d7137fda1" containerName="cleanup"
Sep 10 13:45:16.291190 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.291066    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb697720-d6c8-45db-be71-2635a40605ba" containerName="cleanup"
Sep 10 13:45:16.294084 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.294062    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.302126 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.302104    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-ba18dc\"/\"kube-root-ca.crt\""
Sep 10 13:45:16.302247 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.302107    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-ba18dc\"/\"openshift-service-ca.crt\""
Sep 10 13:45:16.302247 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.302178    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-ba18dc\"/\"default-dockercfg-47rcs\""
Sep 10 13:45:16.302247 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.302177    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-ba18dc\"/\"autores-model-ba18dc-kserve-self-signed-certs\""
Sep 10 13:45:16.312522 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.312502    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"]
Sep 10 13:45:16.407129 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.407100    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-home\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.407129 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.407131    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6a0064-329e-4550-8cc5-3e812fa3615c-tls-certs\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.407312 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.407158    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-tmp-dir\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.407312 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.407184    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-dshm\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.407312 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.407212    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-model-cache\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.407405 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.407318    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnq9s\" (UniqueName: \"kubernetes.io/projected/2c6a0064-329e-4550-8cc5-3e812fa3615c-kube-api-access-mnq9s\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.508661 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.508616    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnq9s\" (UniqueName: \"kubernetes.io/projected/2c6a0064-329e-4550-8cc5-3e812fa3615c-kube-api-access-mnq9s\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.508661 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.508663    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-home\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.508908 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.508681    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6a0064-329e-4550-8cc5-3e812fa3615c-tls-certs\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.508908 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.508701    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-tmp-dir\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.508908 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.508738    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-dshm\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.508908 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.508766    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-model-cache\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.509114 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.509055    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-home\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.509114 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.509082    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-tmp-dir\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.509214 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.509157    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-model-cache\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.510972 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.510950    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-dshm\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.511142 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.511123    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6a0064-329e-4550-8cc5-3e812fa3615c-tls-certs\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.527305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.527278    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnq9s\" (UniqueName: \"kubernetes.io/projected/2c6a0064-329e-4550-8cc5-3e812fa3615c-kube-api-access-mnq9s\") pod \"autores-model-ba18dc-kserve-79dc4f6d6c-5stdt\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") " pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.603097 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.603020    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:16.887841 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.887817    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"]
Sep 10 13:45:16.890169 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:45:16.890143    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c6a0064_329e_4550_8cc5_3e812fa3615c.slice/crio-aa5de47524a9fd96ebd313a1de7fb5f1991919b7e638f0ae305ebbce08762726 WatchSource:0}: Error finding container aa5de47524a9fd96ebd313a1de7fb5f1991919b7e638f0ae305ebbce08762726: Status 404 returned error can't find the container with id aa5de47524a9fd96ebd313a1de7fb5f1991919b7e638f0ae305ebbce08762726
Sep 10 13:45:16.983637 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.983608    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt" event={"ID":"2c6a0064-329e-4550-8cc5-3e812fa3615c","Type":"ContainerStarted","Data":"89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994"}
Sep 10 13:45:16.983775 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:16.983648    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt" event={"ID":"2c6a0064-329e-4550-8cc5-3e812fa3615c","Type":"ContainerStarted","Data":"aa5de47524a9fd96ebd313a1de7fb5f1991919b7e638f0ae305ebbce08762726"}
Sep 10 13:45:17.043461 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:17.043411    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt" podStartSLOduration=1.043394673 podStartE2EDuration="1.043394673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:17.042399382 +0000 UTC m=+875.948957414" watchObservedRunningTime="2026-09-10 13:45:17.043394673 +0000 UTC m=+875.949952703"
Sep 10 13:45:26.093918 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.093860    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"]
Sep 10 13:45:26.099974 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.099953    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.103241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.103206    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f1d168\"/\"default-dockercfg-sc26n\""
Sep 10 13:45:26.104188 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.104161    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f1d168\"/\"kube-root-ca.crt\""
Sep 10 13:45:26.104303 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.104217    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f1d168\"/\"openshift-service-ca.crt\""
Sep 10 13:45:26.104493 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.104443    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f1d168\"/\"test-model-f1d168-kserve-self-signed-certs\""
Sep 10 13:45:26.114958 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.114932    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"]
Sep 10 13:45:26.194767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.194730    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-tmp-dir\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.194949 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.194782    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-dshm\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.194949 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.194911    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-home\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.194949 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.194944    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aad81012-4838-4368-b692-871a497f9f37-tls-certs\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.195052 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.194973    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-model-cache\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.195052 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.195009    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd7x5\" (UniqueName: \"kubernetes.io/projected/aad81012-4838-4368-b692-871a497f9f37-kube-api-access-pd7x5\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.296156 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.296115    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-home\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.296156 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.296158    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aad81012-4838-4368-b692-871a497f9f37-tls-certs\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.296407 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.296193    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-model-cache\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.296407 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.296317    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd7x5\" (UniqueName: \"kubernetes.io/projected/aad81012-4838-4368-b692-871a497f9f37-kube-api-access-pd7x5\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.296407 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.296363    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-tmp-dir\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.296407 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.296409    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-dshm\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.296608 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.296509    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-home\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.296608 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.296567    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-model-cache\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.296758 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.296730    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-tmp-dir\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.298519 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.298496    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-dshm\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.298807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.298788    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aad81012-4838-4368-b692-871a497f9f37-tls-certs\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.306452 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.306432    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd7x5\" (UniqueName: \"kubernetes.io/projected/aad81012-4838-4368-b692-871a497f9f37-kube-api-access-pd7x5\") pod \"test-model-f1d168-kserve-5599fc4589-cjvzw\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") " pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.409560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.409529    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:26.603559 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.603524    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:26.603722 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.603573    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:26.615757 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.615735    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:26.683935 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.683911    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"]
Sep 10 13:45:26.686170 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:45:26.686145    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaad81012_4838_4368_b692_871a497f9f37.slice/crio-c8bc483ee1634e5998423aacf2ecb6d2079928cd01607eca9a7187c77208226d WatchSource:0}: Error finding container c8bc483ee1634e5998423aacf2ecb6d2079928cd01607eca9a7187c77208226d: Status 404 returned error can't find the container with id c8bc483ee1634e5998423aacf2ecb6d2079928cd01607eca9a7187c77208226d
Sep 10 13:45:26.962469 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.962388    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"]
Sep 10 13:45:26.962650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.962622    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz" podUID="c8c9fa08-5fac-4249-baa7-26046e220fb4" containerName="payload-processing" containerID="cri-o://e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68" gracePeriod=30
Sep 10 13:45:26.975427 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.975401    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"]
Sep 10 13:45:26.975607 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:26.975585    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt" podUID="7775ce96-0766-4680-9780-d122fdca46bf" containerName="payload-pre-processing" containerID="cri-o://63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58" gracePeriod=30
Sep 10 13:45:27.030086 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.030050    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw" event={"ID":"aad81012-4838-4368-b692-871a497f9f37","Type":"ContainerStarted","Data":"3a26ce3b737c3d031825bdd9eae4af8ecce36f2e43480ba03d5ddb5f7cc9a76a"}
Sep 10 13:45:27.030238 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.030099    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw" event={"ID":"aad81012-4838-4368-b692-871a497f9f37","Type":"ContainerStarted","Data":"c8bc483ee1634e5998423aacf2ecb6d2079928cd01607eca9a7187c77208226d"}
Sep 10 13:45:27.041073 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.041050    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:45:27.054567 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.054524    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw" podStartSLOduration=1.05450942 podStartE2EDuration="1.05450942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:27.052545671 +0000 UTC m=+885.959103702" watchObservedRunningTime="2026-09-10 13:45:27.05450942 +0000 UTC m=+885.961067492"
Sep 10 13:45:27.229716 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.229687    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:45:27.237455 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.237435    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:45:27.307139 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.307113    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7775ce96-0766-4680-9780-d122fdca46bf-plugins-config-volume\") pod \"7775ce96-0766-4680-9780-d122fdca46bf\" (UID: \"7775ce96-0766-4680-9780-d122fdca46bf\") "
Sep 10 13:45:27.307270 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.307156    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzttd\" (UniqueName: \"kubernetes.io/projected/c8c9fa08-5fac-4249-baa7-26046e220fb4-kube-api-access-xzttd\") pod \"c8c9fa08-5fac-4249-baa7-26046e220fb4\" (UID: \"c8c9fa08-5fac-4249-baa7-26046e220fb4\") "
Sep 10 13:45:27.307329 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.307270    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h67rr\" (UniqueName: \"kubernetes.io/projected/7775ce96-0766-4680-9780-d122fdca46bf-kube-api-access-h67rr\") pod \"7775ce96-0766-4680-9780-d122fdca46bf\" (UID: \"7775ce96-0766-4680-9780-d122fdca46bf\") "
Sep 10 13:45:27.307329 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.307296    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c9fa08-5fac-4249-baa7-26046e220fb4-plugins-config-volume\") pod \"c8c9fa08-5fac-4249-baa7-26046e220fb4\" (UID: \"c8c9fa08-5fac-4249-baa7-26046e220fb4\") "
Sep 10 13:45:27.307551 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.307527    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7775ce96-0766-4680-9780-d122fdca46bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7775ce96-0766-4680-9780-d122fdca46bf" (UID: "7775ce96-0766-4680-9780-d122fdca46bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:45:27.307768 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.307742    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8c9fa08-5fac-4249-baa7-26046e220fb4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c8c9fa08-5fac-4249-baa7-26046e220fb4" (UID: "c8c9fa08-5fac-4249-baa7-26046e220fb4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:45:27.309346 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.309326    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8c9fa08-5fac-4249-baa7-26046e220fb4-kube-api-access-xzttd" (OuterVolumeSpecName: "kube-api-access-xzttd") pod "c8c9fa08-5fac-4249-baa7-26046e220fb4" (UID: "c8c9fa08-5fac-4249-baa7-26046e220fb4"). InnerVolumeSpecName "kube-api-access-xzttd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:45:27.309425 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.309367    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7775ce96-0766-4680-9780-d122fdca46bf-kube-api-access-h67rr" (OuterVolumeSpecName: "kube-api-access-h67rr") pod "7775ce96-0766-4680-9780-d122fdca46bf" (UID: "7775ce96-0766-4680-9780-d122fdca46bf"). InnerVolumeSpecName "kube-api-access-h67rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:45:27.408480 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.408453    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7775ce96-0766-4680-9780-d122fdca46bf-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:45:27.408480 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.408474    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzttd\" (UniqueName: \"kubernetes.io/projected/c8c9fa08-5fac-4249-baa7-26046e220fb4-kube-api-access-xzttd\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:45:27.408480 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.408484    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h67rr\" (UniqueName: \"kubernetes.io/projected/7775ce96-0766-4680-9780-d122fdca46bf-kube-api-access-h67rr\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:45:27.408664 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:27.408492    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c9fa08-5fac-4249-baa7-26046e220fb4-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:45:28.036074 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.035988    2549 generic.go:356] "Generic (PLEG): container finished" podID="7775ce96-0766-4680-9780-d122fdca46bf" containerID="63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58" exitCode=0
Sep 10 13:45:28.036074 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.036031    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"
Sep 10 13:45:28.036277 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.036077    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt" event={"ID":"7775ce96-0766-4680-9780-d122fdca46bf","Type":"ContainerDied","Data":"63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58"}
Sep 10 13:45:28.036277 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.036138    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt" event={"ID":"7775ce96-0766-4680-9780-d122fdca46bf","Type":"ContainerDied","Data":"663e67dcf688cbe4e3d86b410569c201fa4aeef1fd1cd101e4d0a9cbe057de9e"}
Sep 10 13:45:28.036277 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.036171    2549 scope.go:117] "RemoveContainer" containerID="63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58"
Sep 10 13:45:28.037577 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.037482    2549 generic.go:356] "Generic (PLEG): container finished" podID="c8c9fa08-5fac-4249-baa7-26046e220fb4" containerID="e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68" exitCode=0
Sep 10 13:45:28.037577 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.037511    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"
Sep 10 13:45:28.037577 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.037545    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz" event={"ID":"c8c9fa08-5fac-4249-baa7-26046e220fb4","Type":"ContainerDied","Data":"e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68"}
Sep 10 13:45:28.037706 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.037586    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz" event={"ID":"c8c9fa08-5fac-4249-baa7-26046e220fb4","Type":"ContainerDied","Data":"fc6c8d683b8643c5a789f9486cc4c9d1eab5097c4d3dfa7cc8831a499af43947"}
Sep 10 13:45:28.062743 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.062708    2549 scope.go:117] "RemoveContainer" containerID="63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58"
Sep 10 13:45:28.063089 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:45:28.063062    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58\": container with ID starting with 63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58 not found: ID does not exist" containerID="63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58"
Sep 10 13:45:28.063159 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.063098    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58"} err="failed to get container status \"63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58\": rpc error: code = NotFound desc = could not find container \"63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58\": container with ID starting with 63c181f6a1b5749e7269f93f3290b89ca36ed5b7c18cf17589ac07584795fc58 not found: ID does not exist"
Sep 10 13:45:28.063159 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.063114    2549 scope.go:117] "RemoveContainer" containerID="e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68"
Sep 10 13:45:28.071614 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.071597    2549 scope.go:117] "RemoveContainer" containerID="e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68"
Sep 10 13:45:28.071919 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:45:28.071902    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68\": container with ID starting with e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68 not found: ID does not exist" containerID="e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68"
Sep 10 13:45:28.072002 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.071924    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68"} err="failed to get container status \"e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68\": rpc error: code = NotFound desc = could not find container \"e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68\": container with ID starting with e322ed876c3e3644ad9724759bbc7b7c7e988be403cde6541e43b0a162d0ff68 not found: ID does not exist"
Sep 10 13:45:28.084954 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.084932    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"]
Sep 10 13:45:28.089529 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.089509    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0d6a65c4-64499bf5cd-xmvjt"]
Sep 10 13:45:28.150158 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.150130    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"]
Sep 10 13:45:28.151768 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:28.151736    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0d6a65c4-55b7b5bd5-8gspz"]
Sep 10 13:45:29.674221 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:29.674181    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7775ce96-0766-4680-9780-d122fdca46bf" path="/var/lib/kubelet/pods/7775ce96-0766-4680-9780-d122fdca46bf/volumes"
Sep 10 13:45:29.674726 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:29.674706    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8c9fa08-5fac-4249-baa7-26046e220fb4" path="/var/lib/kubelet/pods/c8c9fa08-5fac-4249-baa7-26046e220fb4/volumes"
Sep 10 13:45:36.409738 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:36.409660    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:36.409738 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:36.409708    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:36.421962 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:36.421938    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:37.089737 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:37.089707    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:45:41.533648 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:41.533621    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:45:41.534603 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:45:41.534575    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:46:10.720142 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:10.720111    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"]
Sep 10 13:46:25.739129 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:25.739085    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt" podUID="2c6a0064-329e-4550-8cc5-3e812fa3615c" containerName="main" containerID="cri-o://89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994" gracePeriod=45
Sep 10 13:46:25.975399 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:25.975372    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:46:26.114424 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114318    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6a0064-329e-4550-8cc5-3e812fa3615c-tls-certs\") pod \"2c6a0064-329e-4550-8cc5-3e812fa3615c\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") "
Sep 10 13:46:26.114424 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114402    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnq9s\" (UniqueName: \"kubernetes.io/projected/2c6a0064-329e-4550-8cc5-3e812fa3615c-kube-api-access-mnq9s\") pod \"2c6a0064-329e-4550-8cc5-3e812fa3615c\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") "
Sep 10 13:46:26.114424 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114430    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-dshm\") pod \"2c6a0064-329e-4550-8cc5-3e812fa3615c\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") "
Sep 10 13:46:26.114722 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114447    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-model-cache\") pod \"2c6a0064-329e-4550-8cc5-3e812fa3615c\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") "
Sep 10 13:46:26.114722 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114499    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-tmp-dir\") pod \"2c6a0064-329e-4550-8cc5-3e812fa3615c\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") "
Sep 10 13:46:26.114722 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114549    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-home\") pod \"2c6a0064-329e-4550-8cc5-3e812fa3615c\" (UID: \"2c6a0064-329e-4550-8cc5-3e812fa3615c\") "
Sep 10 13:46:26.114902 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114730    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c6a0064-329e-4550-8cc5-3e812fa3615c" (UID: "2c6a0064-329e-4550-8cc5-3e812fa3615c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:26.114902 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114774    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c6a0064-329e-4550-8cc5-3e812fa3615c" (UID: "2c6a0064-329e-4550-8cc5-3e812fa3615c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:26.114902 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114885    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-home" (OuterVolumeSpecName: "home") pod "2c6a0064-329e-4550-8cc5-3e812fa3615c" (UID: "2c6a0064-329e-4550-8cc5-3e812fa3615c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:26.115127 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114951    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:46:26.115127 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.114970    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:46:26.116589 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.116558    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-dshm" (OuterVolumeSpecName: "dshm") pod "2c6a0064-329e-4550-8cc5-3e812fa3615c" (UID: "2c6a0064-329e-4550-8cc5-3e812fa3615c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:26.116695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.116628    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c6a0064-329e-4550-8cc5-3e812fa3615c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c6a0064-329e-4550-8cc5-3e812fa3615c" (UID: "2c6a0064-329e-4550-8cc5-3e812fa3615c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:46:26.116695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.116653    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c6a0064-329e-4550-8cc5-3e812fa3615c-kube-api-access-mnq9s" (OuterVolumeSpecName: "kube-api-access-mnq9s") pod "2c6a0064-329e-4550-8cc5-3e812fa3615c" (UID: "2c6a0064-329e-4550-8cc5-3e812fa3615c"). InnerVolumeSpecName "kube-api-access-mnq9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:46:26.215411 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.215373    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:46:26.215411 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.215403    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6a0064-329e-4550-8cc5-3e812fa3615c-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:46:26.215411 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.215413    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnq9s\" (UniqueName: \"kubernetes.io/projected/2c6a0064-329e-4550-8cc5-3e812fa3615c-kube-api-access-mnq9s\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:46:26.215627 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.215423    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6a0064-329e-4550-8cc5-3e812fa3615c-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:46:26.300491 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.300460    2549 generic.go:356] "Generic (PLEG): container finished" podID="2c6a0064-329e-4550-8cc5-3e812fa3615c" containerID="89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994" exitCode=0
Sep 10 13:46:26.300650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.300504    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"
Sep 10 13:46:26.300650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.300545    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt" event={"ID":"2c6a0064-329e-4550-8cc5-3e812fa3615c","Type":"ContainerDied","Data":"89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994"}
Sep 10 13:46:26.300650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.300590    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt" event={"ID":"2c6a0064-329e-4550-8cc5-3e812fa3615c","Type":"ContainerDied","Data":"aa5de47524a9fd96ebd313a1de7fb5f1991919b7e638f0ae305ebbce08762726"}
Sep 10 13:46:26.300650 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.300611    2549 scope.go:117] "RemoveContainer" containerID="89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994"
Sep 10 13:46:26.309518 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.309502    2549 scope.go:117] "RemoveContainer" containerID="89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994"
Sep 10 13:46:26.309771 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:46:26.309756    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994\": container with ID starting with 89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994 not found: ID does not exist" containerID="89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994"
Sep 10 13:46:26.309828 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.309779    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994"} err="failed to get container status \"89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994\": rpc error: code = NotFound desc = could not find container \"89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994\": container with ID starting with 89ab7e7399bcfdff28d46fe9f3a753330248af179b1fab21cec58ab9dc599994 not found: ID does not exist"
Sep 10 13:46:26.326280 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.326257    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"]
Sep 10 13:46:26.331954 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:26.331931    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-ba18dc/autores-model-ba18dc-kserve-79dc4f6d6c-5stdt"]
Sep 10 13:46:27.674476 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:27.674442    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c6a0064-329e-4550-8cc5-3e812fa3615c" path="/var/lib/kubelet/pods/2c6a0064-329e-4550-8cc5-3e812fa3615c/volumes"
Sep 10 13:46:49.581270 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.581229    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"]
Sep 10 13:46:49.581695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.581553    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8c9fa08-5fac-4249-baa7-26046e220fb4" containerName="payload-processing"
Sep 10 13:46:49.581695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.581565    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8c9fa08-5fac-4249-baa7-26046e220fb4" containerName="payload-processing"
Sep 10 13:46:49.581695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.581581    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7775ce96-0766-4680-9780-d122fdca46bf" containerName="payload-pre-processing"
Sep 10 13:46:49.581695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.581586    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="7775ce96-0766-4680-9780-d122fdca46bf" containerName="payload-pre-processing"
Sep 10 13:46:49.581695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.581593    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c6a0064-329e-4550-8cc5-3e812fa3615c" containerName="main"
Sep 10 13:46:49.581695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.581599    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c6a0064-329e-4550-8cc5-3e812fa3615c" containerName="main"
Sep 10 13:46:49.581695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.581646    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8c9fa08-5fac-4249-baa7-26046e220fb4" containerName="payload-processing"
Sep 10 13:46:49.581695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.581653    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c6a0064-329e-4550-8cc5-3e812fa3615c" containerName="main"
Sep 10 13:46:49.581695 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.581658    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="7775ce96-0766-4680-9780-d122fdca46bf" containerName="payload-pre-processing"
Sep 10 13:46:49.585749 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.585733    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:46:49.588572 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.588551    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-7eb0a3-dockercfg-vhrt2\""
Sep 10 13:46:49.588811 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.588797    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-7eb0a3\""
Sep 10 13:46:49.597183 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.597161    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"]
Sep 10 13:46:49.699728 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.699692    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b85e274-9e85-4310-9883-709030d1a84a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw\" (UID: \"8b85e274-9e85-4310-9883-709030d1a84a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:46:49.699728 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.699734    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmpsg\" (UniqueName: \"kubernetes.io/projected/8b85e274-9e85-4310-9883-709030d1a84a-kube-api-access-kmpsg\") pod \"payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw\" (UID: \"8b85e274-9e85-4310-9883-709030d1a84a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:46:49.800181 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.800146    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b85e274-9e85-4310-9883-709030d1a84a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw\" (UID: \"8b85e274-9e85-4310-9883-709030d1a84a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:46:49.800327 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.800187    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmpsg\" (UniqueName: \"kubernetes.io/projected/8b85e274-9e85-4310-9883-709030d1a84a-kube-api-access-kmpsg\") pod \"payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw\" (UID: \"8b85e274-9e85-4310-9883-709030d1a84a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:46:49.800850 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.800831    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b85e274-9e85-4310-9883-709030d1a84a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw\" (UID: \"8b85e274-9e85-4310-9883-709030d1a84a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:46:49.810454 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.810426    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmpsg\" (UniqueName: \"kubernetes.io/projected/8b85e274-9e85-4310-9883-709030d1a84a-kube-api-access-kmpsg\") pod \"payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw\" (UID: \"8b85e274-9e85-4310-9883-709030d1a84a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:46:49.896007 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:49.895910    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:46:50.193101 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:46:50.193061    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b85e274_9e85_4310_9883_709030d1a84a.slice/crio-4efdc096db2741cf6184e3be8ae18f27bc12e01fb51a4badb4a6febfd02cd0d9 WatchSource:0}: Error finding container 4efdc096db2741cf6184e3be8ae18f27bc12e01fb51a4badb4a6febfd02cd0d9: Status 404 returned error can't find the container with id 4efdc096db2741cf6184e3be8ae18f27bc12e01fb51a4badb4a6febfd02cd0d9
Sep 10 13:46:50.196378 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:50.196356    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"]
Sep 10 13:46:50.406276 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:50.406239    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw" event={"ID":"8b85e274-9e85-4310-9883-709030d1a84a","Type":"ContainerStarted","Data":"446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241"}
Sep 10 13:46:50.406469 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:50.406295    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw" event={"ID":"8b85e274-9e85-4310-9883-709030d1a84a","Type":"ContainerStarted","Data":"4efdc096db2741cf6184e3be8ae18f27bc12e01fb51a4badb4a6febfd02cd0d9"}
Sep 10 13:46:50.406469 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:50.406326    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:46:50.429385 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:50.429327    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw" podStartSLOduration=1.429309771 podStartE2EDuration="1.429309771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:46:50.428560741 +0000 UTC m=+969.335118769" watchObservedRunningTime="2026-09-10 13:46:50.429309771 +0000 UTC m=+969.335867801"
Sep 10 13:46:56.661390 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:46:56.661356    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"]
Sep 10 13:47:01.412317 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:01.412290    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:47:11.680750 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.680674    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg" podUID="aab3722f-ed15-4e0d-b15b-33001ad1a5d9" containerName="main" containerID="cri-o://a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8" gracePeriod=45
Sep 10 13:47:11.928427 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.928404    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:47:11.992130 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992058    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-home\") pod \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") "
Sep 10 13:47:11.992130 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992089    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-model-cache\") pod \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") "
Sep 10 13:47:11.992130 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992126    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2rp4\" (UniqueName: \"kubernetes.io/projected/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-kube-api-access-x2rp4\") pod \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") "
Sep 10 13:47:11.992375 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992150    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tls-certs\") pod \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") "
Sep 10 13:47:11.992375 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992213    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-dshm\") pod \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") "
Sep 10 13:47:11.992375 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992289    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tmp-dir\") pod \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\" (UID: \"aab3722f-ed15-4e0d-b15b-33001ad1a5d9\") "
Sep 10 13:47:11.992375 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992343    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "aab3722f-ed15-4e0d-b15b-33001ad1a5d9" (UID: "aab3722f-ed15-4e0d-b15b-33001ad1a5d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:11.992375 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992356    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-home" (OuterVolumeSpecName: "home") pod "aab3722f-ed15-4e0d-b15b-33001ad1a5d9" (UID: "aab3722f-ed15-4e0d-b15b-33001ad1a5d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:11.992618 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992526    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aab3722f-ed15-4e0d-b15b-33001ad1a5d9" (UID: "aab3722f-ed15-4e0d-b15b-33001ad1a5d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:11.992618 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992611    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:47:11.992725 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992625    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:47:11.992725 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.992633    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:47:11.994354 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.994322    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aab3722f-ed15-4e0d-b15b-33001ad1a5d9" (UID: "aab3722f-ed15-4e0d-b15b-33001ad1a5d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:47:11.994354 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.994337    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-dshm" (OuterVolumeSpecName: "dshm") pod "aab3722f-ed15-4e0d-b15b-33001ad1a5d9" (UID: "aab3722f-ed15-4e0d-b15b-33001ad1a5d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:11.994527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:11.994363    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-kube-api-access-x2rp4" (OuterVolumeSpecName: "kube-api-access-x2rp4") pod "aab3722f-ed15-4e0d-b15b-33001ad1a5d9" (UID: "aab3722f-ed15-4e0d-b15b-33001ad1a5d9"). InnerVolumeSpecName "kube-api-access-x2rp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:47:12.093769 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.093741    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2rp4\" (UniqueName: \"kubernetes.io/projected/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-kube-api-access-x2rp4\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:47:12.093769 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.093767    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:47:12.093963 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.093777    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aab3722f-ed15-4e0d-b15b-33001ad1a5d9-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:47:12.500525 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.500488    2549 generic.go:356] "Generic (PLEG): container finished" podID="aab3722f-ed15-4e0d-b15b-33001ad1a5d9" containerID="a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8" exitCode=0
Sep 10 13:47:12.500697 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.500542    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg" event={"ID":"aab3722f-ed15-4e0d-b15b-33001ad1a5d9","Type":"ContainerDied","Data":"a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8"}
Sep 10 13:47:12.500697 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.500547    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"
Sep 10 13:47:12.500697 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.500572    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg" event={"ID":"aab3722f-ed15-4e0d-b15b-33001ad1a5d9","Type":"ContainerDied","Data":"a5e63717be52cbf653ab14723e3f2c6ecafca6c7cd73b3fbfc07ea0fccbcb9d4"}
Sep 10 13:47:12.500697 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.500588    2549 scope.go:117] "RemoveContainer" containerID="a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8"
Sep 10 13:47:12.510037 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.510017    2549 scope.go:117] "RemoveContainer" containerID="a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8"
Sep 10 13:47:12.510360 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:47:12.510338    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8\": container with ID starting with a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8 not found: ID does not exist" containerID="a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8"
Sep 10 13:47:12.510435 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.510367    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8"} err="failed to get container status \"a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8\": rpc error: code = NotFound desc = could not find container \"a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8\": container with ID starting with a64a7c1ca372ec152623457dad35b42ad51c0d25b968cf4b6d598dff0ad093f8 not found: ID does not exist"
Sep 10 13:47:12.523948 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.523893    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"]
Sep 10 13:47:12.526448 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:12.526422    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-3696db05-kserve-56ccf74764-vmwrg"]
Sep 10 13:47:13.674074 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:13.674033    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aab3722f-ed15-4e0d-b15b-33001ad1a5d9" path="/var/lib/kubelet/pods/aab3722f-ed15-4e0d-b15b-33001ad1a5d9/volumes"
Sep 10 13:47:27.432922 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.432885    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"]
Sep 10 13:47:27.433331 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.433213    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aab3722f-ed15-4e0d-b15b-33001ad1a5d9" containerName="main"
Sep 10 13:47:27.433331 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.433224    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="aab3722f-ed15-4e0d-b15b-33001ad1a5d9" containerName="main"
Sep 10 13:47:27.433331 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.433279    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="aab3722f-ed15-4e0d-b15b-33001ad1a5d9" containerName="main"
Sep 10 13:47:27.437372 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.437357    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.441364 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.441332    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e6d037\"/\"kube-root-ca.crt\""
Sep 10 13:47:27.442194 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.442175    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e6d037\"/\"default-dockercfg-94x76\""
Sep 10 13:47:27.442284 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.442175    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e6d037\"/\"openshift-service-ca.crt\""
Sep 10 13:47:27.442358 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.442337    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e6d037\"/\"test-model-e6d037-kserve-self-signed-certs\""
Sep 10 13:47:27.454925 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.454906    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"]
Sep 10 13:47:27.531766 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.531728    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdb6k\" (UniqueName: \"kubernetes.io/projected/0b6b0367-0f64-4de2-90ec-dc23c3962478-kube-api-access-cdb6k\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.531944 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.531772    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-model-cache\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.531944 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.531829    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-dshm\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.531944 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.531852    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b6b0367-0f64-4de2-90ec-dc23c3962478-tls-certs\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.531944 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.531916    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-tmp-dir\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.532103 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.531967    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-home\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.632779 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.632745    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-home\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.632982 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.632784    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdb6k\" (UniqueName: \"kubernetes.io/projected/0b6b0367-0f64-4de2-90ec-dc23c3962478-kube-api-access-cdb6k\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.632982 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.632808    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-model-cache\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.632982 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.632847    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-dshm\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.632982 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.632892    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b6b0367-0f64-4de2-90ec-dc23c3962478-tls-certs\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.632982 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.632916    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-tmp-dir\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.633246 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.633213    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-home\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.633246 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.633230    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-model-cache\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.633318 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.633274    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-tmp-dir\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.635134 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.635112    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-dshm\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.635295 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.635279    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b6b0367-0f64-4de2-90ec-dc23c3962478-tls-certs\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.647064 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.647045    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdb6k\" (UniqueName: \"kubernetes.io/projected/0b6b0367-0f64-4de2-90ec-dc23c3962478-kube-api-access-cdb6k\") pod \"test-model-e6d037-kserve-f66d5d4f9-wf4jv\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") " pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:27.746718 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:27.746656    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:28.087646 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:28.087569    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"]
Sep 10 13:47:28.093152 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:47:28.093125    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b6b0367_0f64_4de2_90ec_dc23c3962478.slice/crio-0974bbf854cb2a78e5de4b5dc27ea6353fbe2f5d06cfa2bab3d8e4c2c9bea3b3 WatchSource:0}: Error finding container 0974bbf854cb2a78e5de4b5dc27ea6353fbe2f5d06cfa2bab3d8e4c2c9bea3b3: Status 404 returned error can't find the container with id 0974bbf854cb2a78e5de4b5dc27ea6353fbe2f5d06cfa2bab3d8e4c2c9bea3b3
Sep 10 13:47:28.577811 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:28.577776    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv" event={"ID":"0b6b0367-0f64-4de2-90ec-dc23c3962478","Type":"ContainerStarted","Data":"aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78"}
Sep 10 13:47:28.578157 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:28.577823    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv" event={"ID":"0b6b0367-0f64-4de2-90ec-dc23c3962478","Type":"ContainerStarted","Data":"0974bbf854cb2a78e5de4b5dc27ea6353fbe2f5d06cfa2bab3d8e4c2c9bea3b3"}
Sep 10 13:47:28.617554 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:28.617505    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv" podStartSLOduration=1.6174916160000001 podStartE2EDuration="1.617491616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:47:28.616938592 +0000 UTC m=+1007.523496622" watchObservedRunningTime="2026-09-10 13:47:28.617491616 +0000 UTC m=+1007.524049644"
Sep 10 13:47:37.747139 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:37.747099    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:37.747616 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:37.747159    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:37.759391 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:37.759368    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:47:38.634283 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:47:38.634249    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:48:06.981283 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:06.981251    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"]
Sep 10 13:48:06.981797 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:06.981448    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw" podUID="8b85e274-9e85-4310-9883-709030d1a84a" containerName="payload-processing" containerID="cri-o://446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241" gracePeriod=30
Sep 10 13:48:07.233223 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.233167    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:48:07.370600 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.370568    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmpsg\" (UniqueName: \"kubernetes.io/projected/8b85e274-9e85-4310-9883-709030d1a84a-kube-api-access-kmpsg\") pod \"8b85e274-9e85-4310-9883-709030d1a84a\" (UID: \"8b85e274-9e85-4310-9883-709030d1a84a\") "
Sep 10 13:48:07.370760 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.370675    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b85e274-9e85-4310-9883-709030d1a84a-plugins-config-volume\") pod \"8b85e274-9e85-4310-9883-709030d1a84a\" (UID: \"8b85e274-9e85-4310-9883-709030d1a84a\") "
Sep 10 13:48:07.371072 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.371041    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b85e274-9e85-4310-9883-709030d1a84a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8b85e274-9e85-4310-9883-709030d1a84a" (UID: "8b85e274-9e85-4310-9883-709030d1a84a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:48:07.372530 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.372505    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b85e274-9e85-4310-9883-709030d1a84a-kube-api-access-kmpsg" (OuterVolumeSpecName: "kube-api-access-kmpsg") pod "8b85e274-9e85-4310-9883-709030d1a84a" (UID: "8b85e274-9e85-4310-9883-709030d1a84a"). InnerVolumeSpecName "kube-api-access-kmpsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:48:07.471954 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.471923    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b85e274-9e85-4310-9883-709030d1a84a-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:48:07.471954 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.471949    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmpsg\" (UniqueName: \"kubernetes.io/projected/8b85e274-9e85-4310-9883-709030d1a84a-kube-api-access-kmpsg\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:48:07.745549 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.745518    2549 generic.go:356] "Generic (PLEG): container finished" podID="8b85e274-9e85-4310-9883-709030d1a84a" containerID="446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241" exitCode=0
Sep 10 13:48:07.745700 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.745564    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"
Sep 10 13:48:07.745700 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.745610    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw" event={"ID":"8b85e274-9e85-4310-9883-709030d1a84a","Type":"ContainerDied","Data":"446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241"}
Sep 10 13:48:07.745700 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.745660    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw" event={"ID":"8b85e274-9e85-4310-9883-709030d1a84a","Type":"ContainerDied","Data":"4efdc096db2741cf6184e3be8ae18f27bc12e01fb51a4badb4a6febfd02cd0d9"}
Sep 10 13:48:07.745700 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.745681    2549 scope.go:117] "RemoveContainer" containerID="446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241"
Sep 10 13:48:07.754058 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.754039    2549 scope.go:117] "RemoveContainer" containerID="446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241"
Sep 10 13:48:07.754366 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:48:07.754344    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241\": container with ID starting with 446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241 not found: ID does not exist" containerID="446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241"
Sep 10 13:48:07.754415 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.754378    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241"} err="failed to get container status \"446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241\": rpc error: code = NotFound desc = could not find container \"446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241\": container with ID starting with 446db1dc5d9b3f89801b180d34c4f4c0b587c6ba0bb552776fb72f6dd2a5a241 not found: ID does not exist"
Sep 10 13:48:07.779559 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.779516    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"]
Sep 10 13:48:07.781657 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:07.781638    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7eb0a3-579745d465-ljpdw"]
Sep 10 13:48:09.673532 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:09.673502    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b85e274-9e85-4310-9883-709030d1a84a" path="/var/lib/kubelet/pods/8b85e274-9e85-4310-9883-709030d1a84a/volumes"
Sep 10 13:48:42.199117 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.199087    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"]
Sep 10 13:48:42.199482 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.199420    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b85e274-9e85-4310-9883-709030d1a84a" containerName="payload-processing"
Sep 10 13:48:42.199482 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.199431    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b85e274-9e85-4310-9883-709030d1a84a" containerName="payload-processing"
Sep 10 13:48:42.199582 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.199489    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b85e274-9e85-4310-9883-709030d1a84a" containerName="payload-processing"
Sep 10 13:48:42.203559 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.203536    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.207987 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.207963    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-33cc41\"/\"noten-model-33cc41-kserve-self-signed-certs\""
Sep 10 13:48:42.208118 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.207971    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-33cc41\"/\"kube-root-ca.crt\""
Sep 10 13:48:42.208118 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.208034    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-33cc41\"/\"openshift-service-ca.crt\""
Sep 10 13:48:42.208240 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.208120    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-33cc41\"/\"default-dockercfg-xv2rz\""
Sep 10 13:48:42.215208 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.215186    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"]
Sep 10 13:48:42.269402 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.269376    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-home\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.269566 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.269424    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tls-certs\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.269566 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.269490    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-model-cache\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.269566 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.269538    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-dshm\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.269566 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.269564    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvkxf\" (UniqueName: \"kubernetes.io/projected/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-kube-api-access-mvkxf\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.269733 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.269618    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tmp-dir\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.370731 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.370703    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tls-certs\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.370937 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.370756    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-model-cache\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.370937 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.370799    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-dshm\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.370937 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.370825    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvkxf\" (UniqueName: \"kubernetes.io/projected/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-kube-api-access-mvkxf\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.370937 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.370890    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tmp-dir\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.370937 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.370935    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-home\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.371248 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.371208    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-model-cache\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.371367 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.371282    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-home\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.371367 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.371316    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tmp-dir\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.373023 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.373002    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-dshm\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.373245 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.373229    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tls-certs\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.382693 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.382675    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvkxf\" (UniqueName: \"kubernetes.io/projected/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-kube-api-access-mvkxf\") pod \"noten-model-33cc41-kserve-6f8bc4c87f-6hv6h\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") " pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.512220 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.512141    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:42.786915 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.786826    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"]
Sep 10 13:48:42.789246 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:48:42.789220    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7df53c79_59d7_4c05_a5d8_6eb75b37abc1.slice/crio-45918b9f8e838649d969fc697c5129506f4839dc7d8bff33379adddc84289af2 WatchSource:0}: Error finding container 45918b9f8e838649d969fc697c5129506f4839dc7d8bff33379adddc84289af2: Status 404 returned error can't find the container with id 45918b9f8e838649d969fc697c5129506f4839dc7d8bff33379adddc84289af2
Sep 10 13:48:42.903275 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.903244    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h" event={"ID":"7df53c79-59d7-4c05-a5d8-6eb75b37abc1","Type":"ContainerStarted","Data":"e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510"}
Sep 10 13:48:42.903275 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.903280    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h" event={"ID":"7df53c79-59d7-4c05-a5d8-6eb75b37abc1","Type":"ContainerStarted","Data":"45918b9f8e838649d969fc697c5129506f4839dc7d8bff33379adddc84289af2"}
Sep 10 13:48:42.928271 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:42.928137    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h" podStartSLOduration=0.928090705 podStartE2EDuration="928.090705ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:48:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:48:42.926647859 +0000 UTC m=+1081.833205889" watchObservedRunningTime="2026-09-10 13:48:42.928090705 +0000 UTC m=+1081.834648736"
Sep 10 13:48:50.851717 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:50.851683    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"]
Sep 10 13:48:51.921632 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:51.921598    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"]
Sep 10 13:48:52.512957 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:52.512910    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:48:53.105063 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:48:53.105034    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"]
Sep 10 13:49:04.305974 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.305945    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"]
Sep 10 13:49:04.306358 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.306135    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg" podUID="60a72794-0b71-4ed3-b35a-cfecf2e07a3f" containerName="payload-processing" containerID="cri-o://f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc" gracePeriod=30
Sep 10 13:49:04.328509 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.328485    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"]
Sep 10 13:49:04.328721 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.328689    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9" podUID="33d5347d-2814-48e4-8fa1-0a93113e514b" containerName="payload-pre-processing" containerID="cri-o://479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134" gracePeriod=30
Sep 10 13:49:04.595273 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.595251    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:49:04.597907 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.597888    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:49:04.758751 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.758718    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq9g8\" (UniqueName: \"kubernetes.io/projected/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-kube-api-access-vq9g8\") pod \"60a72794-0b71-4ed3-b35a-cfecf2e07a3f\" (UID: \"60a72794-0b71-4ed3-b35a-cfecf2e07a3f\") "
Sep 10 13:49:04.758990 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.758784    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tvnb\" (UniqueName: \"kubernetes.io/projected/33d5347d-2814-48e4-8fa1-0a93113e514b-kube-api-access-5tvnb\") pod \"33d5347d-2814-48e4-8fa1-0a93113e514b\" (UID: \"33d5347d-2814-48e4-8fa1-0a93113e514b\") "
Sep 10 13:49:04.758990 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.758827    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d5347d-2814-48e4-8fa1-0a93113e514b-plugins-config-volume\") pod \"33d5347d-2814-48e4-8fa1-0a93113e514b\" (UID: \"33d5347d-2814-48e4-8fa1-0a93113e514b\") "
Sep 10 13:49:04.758990 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.758855    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-plugins-config-volume\") pod \"60a72794-0b71-4ed3-b35a-cfecf2e07a3f\" (UID: \"60a72794-0b71-4ed3-b35a-cfecf2e07a3f\") "
Sep 10 13:49:04.759268 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.759238    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33d5347d-2814-48e4-8fa1-0a93113e514b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "33d5347d-2814-48e4-8fa1-0a93113e514b" (UID: "33d5347d-2814-48e4-8fa1-0a93113e514b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:49:04.759268 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.759241    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "60a72794-0b71-4ed3-b35a-cfecf2e07a3f" (UID: "60a72794-0b71-4ed3-b35a-cfecf2e07a3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:49:04.760912 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.760883    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33d5347d-2814-48e4-8fa1-0a93113e514b-kube-api-access-5tvnb" (OuterVolumeSpecName: "kube-api-access-5tvnb") pod "33d5347d-2814-48e4-8fa1-0a93113e514b" (UID: "33d5347d-2814-48e4-8fa1-0a93113e514b"). InnerVolumeSpecName "kube-api-access-5tvnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:49:04.761007 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.760909    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-kube-api-access-vq9g8" (OuterVolumeSpecName: "kube-api-access-vq9g8") pod "60a72794-0b71-4ed3-b35a-cfecf2e07a3f" (UID: "60a72794-0b71-4ed3-b35a-cfecf2e07a3f"). InnerVolumeSpecName "kube-api-access-vq9g8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:49:04.859478 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.859407    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d5347d-2814-48e4-8fa1-0a93113e514b-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:04.859478 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.859431    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:04.859478 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.859444    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq9g8\" (UniqueName: \"kubernetes.io/projected/60a72794-0b71-4ed3-b35a-cfecf2e07a3f-kube-api-access-vq9g8\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:04.859478 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.859455    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tvnb\" (UniqueName: \"kubernetes.io/projected/33d5347d-2814-48e4-8fa1-0a93113e514b-kube-api-access-5tvnb\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:04.995757 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.995721    2549 generic.go:356] "Generic (PLEG): container finished" podID="60a72794-0b71-4ed3-b35a-cfecf2e07a3f" containerID="f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc" exitCode=0
Sep 10 13:49:04.995953 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.995762    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"
Sep 10 13:49:04.995953 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.995812    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg" event={"ID":"60a72794-0b71-4ed3-b35a-cfecf2e07a3f","Type":"ContainerDied","Data":"f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc"}
Sep 10 13:49:04.995953 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.995910    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg" event={"ID":"60a72794-0b71-4ed3-b35a-cfecf2e07a3f","Type":"ContainerDied","Data":"80fc909cc0404600037108dd8db467710732096f9d6aae7cd10e4d3d14b873f8"}
Sep 10 13:49:04.995953 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.995952    2549 scope.go:117] "RemoveContainer" containerID="f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc"
Sep 10 13:49:04.997157 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.997137    2549 generic.go:356] "Generic (PLEG): container finished" podID="33d5347d-2814-48e4-8fa1-0a93113e514b" containerID="479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134" exitCode=0
Sep 10 13:49:04.997223 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.997193    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"
Sep 10 13:49:04.997223 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.997205    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9" event={"ID":"33d5347d-2814-48e4-8fa1-0a93113e514b","Type":"ContainerDied","Data":"479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134"}
Sep 10 13:49:04.997302 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:04.997237    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9" event={"ID":"33d5347d-2814-48e4-8fa1-0a93113e514b","Type":"ContainerDied","Data":"969e361790727eea50679b05beda1971a3bb1a36063faa736a48f6e0330415ed"}
Sep 10 13:49:05.004232 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.004214    2549 scope.go:117] "RemoveContainer" containerID="f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc"
Sep 10 13:49:05.004501 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:49:05.004479    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc\": container with ID starting with f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc not found: ID does not exist" containerID="f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc"
Sep 10 13:49:05.004603 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.004512    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc"} err="failed to get container status \"f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc\": rpc error: code = NotFound desc = could not find container \"f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc\": container with ID starting with f0b0b409f4c4e1f257f3e2c152c852a0066d2dd4fc907a5c52438254501ed0bc not found: ID does not exist"
Sep 10 13:49:05.004603 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.004535    2549 scope.go:117] "RemoveContainer" containerID="479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134"
Sep 10 13:49:05.012062 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.012048    2549 scope.go:117] "RemoveContainer" containerID="479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134"
Sep 10 13:49:05.012302 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:49:05.012284    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134\": container with ID starting with 479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134 not found: ID does not exist" containerID="479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134"
Sep 10 13:49:05.012356 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.012306    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134"} err="failed to get container status \"479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134\": rpc error: code = NotFound desc = could not find container \"479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134\": container with ID starting with 479f3521dfa0440359cca79af3490aa8631c11bc72e99fd3530e5c44336e3134 not found: ID does not exist"
Sep 10 13:49:05.048723 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.048696    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"]
Sep 10 13:49:05.057069 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.057040    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f1d168-59ccb6f95d-wcfl9"]
Sep 10 13:49:05.090513 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.090494    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"]
Sep 10 13:49:05.106453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.104174    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f1d168-5646bf4fd9-cncfg"]
Sep 10 13:49:05.673194 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.673165    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33d5347d-2814-48e4-8fa1-0a93113e514b" path="/var/lib/kubelet/pods/33d5347d-2814-48e4-8fa1-0a93113e514b/volumes"
Sep 10 13:49:05.673543 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.673496    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60a72794-0b71-4ed3-b35a-cfecf2e07a3f" path="/var/lib/kubelet/pods/60a72794-0b71-4ed3-b35a-cfecf2e07a3f/volumes"
Sep 10 13:49:05.870520 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:05.870480    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw" podUID="aad81012-4838-4368-b692-871a497f9f37" containerName="main" containerID="cri-o://3a26ce3b737c3d031825bdd9eae4af8ecce36f2e43480ba03d5ddb5f7cc9a76a" gracePeriod=45
Sep 10 13:49:06.004644 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.004615    2549 generic.go:356] "Generic (PLEG): container finished" podID="aad81012-4838-4368-b692-871a497f9f37" containerID="3a26ce3b737c3d031825bdd9eae4af8ecce36f2e43480ba03d5ddb5f7cc9a76a" exitCode=0
Sep 10 13:49:06.004766 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.004682    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw" event={"ID":"aad81012-4838-4368-b692-871a497f9f37","Type":"ContainerDied","Data":"3a26ce3b737c3d031825bdd9eae4af8ecce36f2e43480ba03d5ddb5f7cc9a76a"}
Sep 10 13:49:06.117162 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.117138    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:49:06.272238 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272156    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-model-cache\") pod \"aad81012-4838-4368-b692-871a497f9f37\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") "
Sep 10 13:49:06.272238 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272205    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aad81012-4838-4368-b692-871a497f9f37-tls-certs\") pod \"aad81012-4838-4368-b692-871a497f9f37\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") "
Sep 10 13:49:06.272238 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272234    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-dshm\") pod \"aad81012-4838-4368-b692-871a497f9f37\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") "
Sep 10 13:49:06.272496 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272270    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-tmp-dir\") pod \"aad81012-4838-4368-b692-871a497f9f37\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") "
Sep 10 13:49:06.272496 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272331    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd7x5\" (UniqueName: \"kubernetes.io/projected/aad81012-4838-4368-b692-871a497f9f37-kube-api-access-pd7x5\") pod \"aad81012-4838-4368-b692-871a497f9f37\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") "
Sep 10 13:49:06.272496 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272401    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-home\") pod \"aad81012-4838-4368-b692-871a497f9f37\" (UID: \"aad81012-4838-4368-b692-871a497f9f37\") "
Sep 10 13:49:06.272496 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272435    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-model-cache" (OuterVolumeSpecName: "model-cache") pod "aad81012-4838-4368-b692-871a497f9f37" (UID: "aad81012-4838-4368-b692-871a497f9f37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:06.272662 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272577    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aad81012-4838-4368-b692-871a497f9f37" (UID: "aad81012-4838-4368-b692-871a497f9f37"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:06.272700 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272662    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-home" (OuterVolumeSpecName: "home") pod "aad81012-4838-4368-b692-871a497f9f37" (UID: "aad81012-4838-4368-b692-871a497f9f37"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:06.272734 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272704    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:06.272734 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.272722    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:06.274505 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.274481    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aad81012-4838-4368-b692-871a497f9f37-kube-api-access-pd7x5" (OuterVolumeSpecName: "kube-api-access-pd7x5") pod "aad81012-4838-4368-b692-871a497f9f37" (UID: "aad81012-4838-4368-b692-871a497f9f37"). InnerVolumeSpecName "kube-api-access-pd7x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:49:06.274714 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.274693    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aad81012-4838-4368-b692-871a497f9f37-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aad81012-4838-4368-b692-871a497f9f37" (UID: "aad81012-4838-4368-b692-871a497f9f37"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:49:06.274770 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.274730    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-dshm" (OuterVolumeSpecName: "dshm") pod "aad81012-4838-4368-b692-871a497f9f37" (UID: "aad81012-4838-4368-b692-871a497f9f37"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:06.373992 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.373967    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:06.373992 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.373989    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aad81012-4838-4368-b692-871a497f9f37-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:06.373992 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.373999    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aad81012-4838-4368-b692-871a497f9f37-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:06.374207 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.374008    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd7x5\" (UniqueName: \"kubernetes.io/projected/aad81012-4838-4368-b692-871a497f9f37-kube-api-access-pd7x5\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:06.940792 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:06.940754    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h" podUID="7df53c79-59d7-4c05-a5d8-6eb75b37abc1" containerName="main" containerID="cri-o://e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510" gracePeriod=45
Sep 10 13:49:07.010714 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.010685    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw" event={"ID":"aad81012-4838-4368-b692-871a497f9f37","Type":"ContainerDied","Data":"c8bc483ee1634e5998423aacf2ecb6d2079928cd01607eca9a7187c77208226d"}
Sep 10 13:49:07.010828 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.010712    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"
Sep 10 13:49:07.010828 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.010729    2549 scope.go:117] "RemoveContainer" containerID="3a26ce3b737c3d031825bdd9eae4af8ecce36f2e43480ba03d5ddb5f7cc9a76a"
Sep 10 13:49:07.044185 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.044160    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"]
Sep 10 13:49:07.055766 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.055738    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-f1d168/test-model-f1d168-kserve-5599fc4589-cjvzw"]
Sep 10 13:49:07.179300 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.179277    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:49:07.283055 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.282986    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-home\") pod \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") "
Sep 10 13:49:07.283055 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.283021    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-model-cache\") pod \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") "
Sep 10 13:49:07.283055 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.283036    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-dshm\") pod \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") "
Sep 10 13:49:07.283294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.283063    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tls-certs\") pod \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") "
Sep 10 13:49:07.283294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.283085    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tmp-dir\") pod \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") "
Sep 10 13:49:07.283294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.283110    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvkxf\" (UniqueName: \"kubernetes.io/projected/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-kube-api-access-mvkxf\") pod \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\" (UID: \"7df53c79-59d7-4c05-a5d8-6eb75b37abc1\") "
Sep 10 13:49:07.283294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.283283    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-model-cache" (OuterVolumeSpecName: "model-cache") pod "7df53c79-59d7-4c05-a5d8-6eb75b37abc1" (UID: "7df53c79-59d7-4c05-a5d8-6eb75b37abc1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:07.283466 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.283296    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-home" (OuterVolumeSpecName: "home") pod "7df53c79-59d7-4c05-a5d8-6eb75b37abc1" (UID: "7df53c79-59d7-4c05-a5d8-6eb75b37abc1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:07.283466 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.283399    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7df53c79-59d7-4c05-a5d8-6eb75b37abc1" (UID: "7df53c79-59d7-4c05-a5d8-6eb75b37abc1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:07.285253 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.285223    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-kube-api-access-mvkxf" (OuterVolumeSpecName: "kube-api-access-mvkxf") pod "7df53c79-59d7-4c05-a5d8-6eb75b37abc1" (UID: "7df53c79-59d7-4c05-a5d8-6eb75b37abc1"). InnerVolumeSpecName "kube-api-access-mvkxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:49:07.285348 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.285273    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-dshm" (OuterVolumeSpecName: "dshm") pod "7df53c79-59d7-4c05-a5d8-6eb75b37abc1" (UID: "7df53c79-59d7-4c05-a5d8-6eb75b37abc1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:07.285348 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.285272    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7df53c79-59d7-4c05-a5d8-6eb75b37abc1" (UID: "7df53c79-59d7-4c05-a5d8-6eb75b37abc1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:49:07.384260 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.384223    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:07.384260 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.384255    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:07.384260 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.384266    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:07.384260 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.384273    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:07.384474 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.384281    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:07.384474 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.384289    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvkxf\" (UniqueName: \"kubernetes.io/projected/7df53c79-59d7-4c05-a5d8-6eb75b37abc1-kube-api-access-mvkxf\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:07.674090 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:07.674057    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aad81012-4838-4368-b692-871a497f9f37" path="/var/lib/kubelet/pods/aad81012-4838-4368-b692-871a497f9f37/volumes"
Sep 10 13:49:08.015894 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.015795    2549 generic.go:356] "Generic (PLEG): container finished" podID="7df53c79-59d7-4c05-a5d8-6eb75b37abc1" containerID="e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510" exitCode=0
Sep 10 13:49:08.015894 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.015834    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"
Sep 10 13:49:08.016320 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.015890    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h" event={"ID":"7df53c79-59d7-4c05-a5d8-6eb75b37abc1","Type":"ContainerDied","Data":"e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510"}
Sep 10 13:49:08.016320 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.015930    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h" event={"ID":"7df53c79-59d7-4c05-a5d8-6eb75b37abc1","Type":"ContainerDied","Data":"45918b9f8e838649d969fc697c5129506f4839dc7d8bff33379adddc84289af2"}
Sep 10 13:49:08.016320 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.015947    2549 scope.go:117] "RemoveContainer" containerID="e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510"
Sep 10 13:49:08.024019 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.024002    2549 scope.go:117] "RemoveContainer" containerID="e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510"
Sep 10 13:49:08.024289 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:49:08.024271    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510\": container with ID starting with e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510 not found: ID does not exist" containerID="e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510"
Sep 10 13:49:08.024343 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.024298    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510"} err="failed to get container status \"e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510\": rpc error: code = NotFound desc = could not find container \"e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510\": container with ID starting with e7e4e8898ace774ee70308650e11022359c4140b2dd58b2c54fd278a6005e510 not found: ID does not exist"
Sep 10 13:49:08.042251 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.042228    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"]
Sep 10 13:49:08.048350 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.048328    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-33cc41/noten-model-33cc41-kserve-6f8bc4c87f-6hv6h"]
Sep 10 13:49:08.123313 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.123281    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv" podUID="0b6b0367-0f64-4de2-90ec-dc23c3962478" containerName="main" containerID="cri-o://aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78" gracePeriod=45
Sep 10 13:49:08.377998 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.377977    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:49:08.493642 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.493613    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-home\") pod \"0b6b0367-0f64-4de2-90ec-dc23c3962478\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") "
Sep 10 13:49:08.493798 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.493661    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-dshm\") pod \"0b6b0367-0f64-4de2-90ec-dc23c3962478\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") "
Sep 10 13:49:08.493798 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.493691    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdb6k\" (UniqueName: \"kubernetes.io/projected/0b6b0367-0f64-4de2-90ec-dc23c3962478-kube-api-access-cdb6k\") pod \"0b6b0367-0f64-4de2-90ec-dc23c3962478\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") "
Sep 10 13:49:08.493798 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.493714    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-model-cache\") pod \"0b6b0367-0f64-4de2-90ec-dc23c3962478\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") "
Sep 10 13:49:08.493798 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.493730    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b6b0367-0f64-4de2-90ec-dc23c3962478-tls-certs\") pod \"0b6b0367-0f64-4de2-90ec-dc23c3962478\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") "
Sep 10 13:49:08.493798 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.493750    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-tmp-dir\") pod \"0b6b0367-0f64-4de2-90ec-dc23c3962478\" (UID: \"0b6b0367-0f64-4de2-90ec-dc23c3962478\") "
Sep 10 13:49:08.494077 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.493934    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-home" (OuterVolumeSpecName: "home") pod "0b6b0367-0f64-4de2-90ec-dc23c3962478" (UID: "0b6b0367-0f64-4de2-90ec-dc23c3962478"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:08.494077 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.494012    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-model-cache" (OuterVolumeSpecName: "model-cache") pod "0b6b0367-0f64-4de2-90ec-dc23c3962478" (UID: "0b6b0367-0f64-4de2-90ec-dc23c3962478"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:08.494171 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.494127    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0b6b0367-0f64-4de2-90ec-dc23c3962478" (UID: "0b6b0367-0f64-4de2-90ec-dc23c3962478"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:08.495941 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.495913    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-dshm" (OuterVolumeSpecName: "dshm") pod "0b6b0367-0f64-4de2-90ec-dc23c3962478" (UID: "0b6b0367-0f64-4de2-90ec-dc23c3962478"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:08.496031 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.495969    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b6b0367-0f64-4de2-90ec-dc23c3962478-kube-api-access-cdb6k" (OuterVolumeSpecName: "kube-api-access-cdb6k") pod "0b6b0367-0f64-4de2-90ec-dc23c3962478" (UID: "0b6b0367-0f64-4de2-90ec-dc23c3962478"). InnerVolumeSpecName "kube-api-access-cdb6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:49:08.496031 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.495972    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b6b0367-0f64-4de2-90ec-dc23c3962478-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0b6b0367-0f64-4de2-90ec-dc23c3962478" (UID: "0b6b0367-0f64-4de2-90ec-dc23c3962478"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:49:08.595259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.595192    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:08.595259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.595214    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:08.595259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.595224    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdb6k\" (UniqueName: \"kubernetes.io/projected/0b6b0367-0f64-4de2-90ec-dc23c3962478-kube-api-access-cdb6k\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:08.595259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.595232    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:08.595259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.595240    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b6b0367-0f64-4de2-90ec-dc23c3962478-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:08.595259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:08.595247    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6b0367-0f64-4de2-90ec-dc23c3962478-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:09.021945 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.021914    2549 generic.go:356] "Generic (PLEG): container finished" podID="0b6b0367-0f64-4de2-90ec-dc23c3962478" containerID="aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78" exitCode=0
Sep 10 13:49:09.022527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.021954    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"
Sep 10 13:49:09.022527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.021993    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv" event={"ID":"0b6b0367-0f64-4de2-90ec-dc23c3962478","Type":"ContainerDied","Data":"aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78"}
Sep 10 13:49:09.022527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.022031    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv" event={"ID":"0b6b0367-0f64-4de2-90ec-dc23c3962478","Type":"ContainerDied","Data":"0974bbf854cb2a78e5de4b5dc27ea6353fbe2f5d06cfa2bab3d8e4c2c9bea3b3"}
Sep 10 13:49:09.022527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.022052    2549 scope.go:117] "RemoveContainer" containerID="aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78"
Sep 10 13:49:09.030845 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.030827    2549 scope.go:117] "RemoveContainer" containerID="aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78"
Sep 10 13:49:09.031162 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:49:09.031142    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78\": container with ID starting with aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78 not found: ID does not exist" containerID="aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78"
Sep 10 13:49:09.031206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.031170    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78"} err="failed to get container status \"aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78\": rpc error: code = NotFound desc = could not find container \"aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78\": container with ID starting with aef47ea4b08a130dddd7ee0a10d2da8fc35db4ade6c91c38369dafb86def5b78 not found: ID does not exist"
Sep 10 13:49:09.051322 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.051294    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"]
Sep 10 13:49:09.058060 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.058038    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e6d037/test-model-e6d037-kserve-f66d5d4f9-wf4jv"]
Sep 10 13:49:09.673928 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.673889    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b6b0367-0f64-4de2-90ec-dc23c3962478" path="/var/lib/kubelet/pods/0b6b0367-0f64-4de2-90ec-dc23c3962478/volumes"
Sep 10 13:49:09.674359 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:09.674339    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7df53c79-59d7-4c05-a5d8-6eb75b37abc1" path="/var/lib/kubelet/pods/7df53c79-59d7-4c05-a5d8-6eb75b37abc1/volumes"
Sep 10 13:49:12.790561 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790523    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"]
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790839    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aad81012-4838-4368-b692-871a497f9f37" containerName="main"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790851    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="aad81012-4838-4368-b692-871a497f9f37" containerName="main"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790877    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df53c79-59d7-4c05-a5d8-6eb75b37abc1" containerName="main"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790883    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df53c79-59d7-4c05-a5d8-6eb75b37abc1" containerName="main"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790892    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b6b0367-0f64-4de2-90ec-dc23c3962478" containerName="main"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790897    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b6b0367-0f64-4de2-90ec-dc23c3962478" containerName="main"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790923    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33d5347d-2814-48e4-8fa1-0a93113e514b" containerName="payload-pre-processing"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790929    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="33d5347d-2814-48e4-8fa1-0a93113e514b" containerName="payload-pre-processing"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790936    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a72794-0b71-4ed3-b35a-cfecf2e07a3f" containerName="payload-processing"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790941    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a72794-0b71-4ed3-b35a-cfecf2e07a3f" containerName="payload-processing"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790988    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="60a72794-0b71-4ed3-b35a-cfecf2e07a3f" containerName="payload-processing"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.790997    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b6b0367-0f64-4de2-90ec-dc23c3962478" containerName="main"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.791003    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="aad81012-4838-4368-b692-871a497f9f37" containerName="main"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.791009    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="33d5347d-2814-48e4-8fa1-0a93113e514b" containerName="payload-pre-processing"
Sep 10 13:49:12.791241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.791016    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df53c79-59d7-4c05-a5d8-6eb75b37abc1" containerName="main"
Sep 10 13:49:12.794161 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.794142    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"
Sep 10 13:49:12.796596 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.796568    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 13:49:12.796709 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.796670    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:49:12.803008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.802978    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"]
Sep 10 13:49:12.827206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.827181    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3b8613-2548-45f8-9b8f-033261e72e3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m\" (UID: \"4f3b8613-2548-45f8-9b8f-033261e72e3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"
Sep 10 13:49:12.928490 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.928452    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3b8613-2548-45f8-9b8f-033261e72e3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m\" (UID: \"4f3b8613-2548-45f8-9b8f-033261e72e3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"
Sep 10 13:49:12.929139 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:12.929113    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3b8613-2548-45f8-9b8f-033261e72e3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m\" (UID: \"4f3b8613-2548-45f8-9b8f-033261e72e3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"
Sep 10 13:49:13.105132 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:13.105043    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"
Sep 10 13:49:13.384816 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:13.384737    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"]
Sep 10 13:49:13.387478 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:49:13.387451    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f3b8613_2548_45f8_9b8f_033261e72e3e.slice/crio-da6bd57a013051087db7120163cc0c36074690ca54e5602e34144e5207f351e6 WatchSource:0}: Error finding container da6bd57a013051087db7120163cc0c36074690ca54e5602e34144e5207f351e6: Status 404 returned error can't find the container with id da6bd57a013051087db7120163cc0c36074690ca54e5602e34144e5207f351e6
Sep 10 13:49:14.046946 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:14.046917    2549 generic.go:356] "Generic (PLEG): container finished" podID="4f3b8613-2548-45f8-9b8f-033261e72e3e" containerID="361fe5762bdd43a18ea62ae4416730fc6bb3f9170dfed50b15cc5557a9c03cb8" exitCode=0
Sep 10 13:49:14.047241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:14.046963    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m" event={"ID":"4f3b8613-2548-45f8-9b8f-033261e72e3e","Type":"ContainerDied","Data":"361fe5762bdd43a18ea62ae4416730fc6bb3f9170dfed50b15cc5557a9c03cb8"}
Sep 10 13:49:14.047241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:14.047004    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m" event={"ID":"4f3b8613-2548-45f8-9b8f-033261e72e3e","Type":"ContainerStarted","Data":"da6bd57a013051087db7120163cc0c36074690ca54e5602e34144e5207f351e6"}
Sep 10 13:49:15.172011 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:15.171987    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"
Sep 10 13:49:15.245810 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:15.245782    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3b8613-2548-45f8-9b8f-033261e72e3e-maas-api-service-ca\") pod \"4f3b8613-2548-45f8-9b8f-033261e72e3e\" (UID: \"4f3b8613-2548-45f8-9b8f-033261e72e3e\") "
Sep 10 13:49:15.246154 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:15.246131    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f3b8613-2548-45f8-9b8f-033261e72e3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f3b8613-2548-45f8-9b8f-033261e72e3e" (UID: "4f3b8613-2548-45f8-9b8f-033261e72e3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:49:15.346610 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:15.346531    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3b8613-2548-45f8-9b8f-033261e72e3e-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:16.056186 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:16.056100    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"
Sep 10 13:49:16.056186 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:16.056133    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m" event={"ID":"4f3b8613-2548-45f8-9b8f-033261e72e3e","Type":"ContainerDied","Data":"da6bd57a013051087db7120163cc0c36074690ca54e5602e34144e5207f351e6"}
Sep 10 13:49:16.056186 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:16.056178    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da6bd57a013051087db7120163cc0c36074690ca54e5602e34144e5207f351e6"
Sep 10 13:49:21.236758 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.236721    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"]
Sep 10 13:49:21.237148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.237046    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f3b8613-2548-45f8-9b8f-033261e72e3e" containerName="revoke-keys"
Sep 10 13:49:21.237148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.237058    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f3b8613-2548-45f8-9b8f-033261e72e3e" containerName="revoke-keys"
Sep 10 13:49:21.237148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.237109    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f3b8613-2548-45f8-9b8f-033261e72e3e" containerName="revoke-keys"
Sep 10 13:49:21.240034 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.240019    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:49:21.242505 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.242481    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-3d0caa-dockercfg-xghjr\""
Sep 10 13:49:21.242633 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.242481    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-3d0caa\""
Sep 10 13:49:21.258620 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.258599    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"]
Sep 10 13:49:21.298546 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.298516    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86\" (UID: \"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:49:21.298691 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.298557    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9998\" (UniqueName: \"kubernetes.io/projected/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-kube-api-access-k9998\") pod \"payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86\" (UID: \"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:49:21.399205 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.399167    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86\" (UID: \"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:49:21.399376 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.399214    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9998\" (UniqueName: \"kubernetes.io/projected/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-kube-api-access-k9998\") pod \"payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86\" (UID: \"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:49:21.399917 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.399895    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86\" (UID: \"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:49:21.410356 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.410333    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9998\" (UniqueName: \"kubernetes.io/projected/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-kube-api-access-k9998\") pod \"payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86\" (UID: \"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:49:21.550091 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.550018    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:49:21.825176 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:21.825103    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"]
Sep 10 13:49:21.828666 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:49:21.828641    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode245b3c7_0ead_4b10_ab64_bedfa8b89a5e.slice/crio-931c130ca3d691374f80cc7af4c8b4a9bf139b7e3541dd811c3e04cdfd6122e3 WatchSource:0}: Error finding container 931c130ca3d691374f80cc7af4c8b4a9bf139b7e3541dd811c3e04cdfd6122e3: Status 404 returned error can't find the container with id 931c130ca3d691374f80cc7af4c8b4a9bf139b7e3541dd811c3e04cdfd6122e3
Sep 10 13:49:22.085392 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:22.085308    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86" event={"ID":"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e","Type":"ContainerStarted","Data":"941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d"}
Sep 10 13:49:22.085392 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:22.085354    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86" event={"ID":"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e","Type":"ContainerStarted","Data":"931c130ca3d691374f80cc7af4c8b4a9bf139b7e3541dd811c3e04cdfd6122e3"}
Sep 10 13:49:22.085604 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:22.085429    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:49:22.113181 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:22.113133    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86" podStartSLOduration=1.113119473 podStartE2EDuration="1.113119473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:49:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:49:22.111528278 +0000 UTC m=+1121.018086320" watchObservedRunningTime="2026-09-10 13:49:22.113119473 +0000 UTC m=+1121.019677502"
Sep 10 13:49:23.418570 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.418541    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w"]
Sep 10 13:49:23.423509 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.423483    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e6d037-29817465-q4j6w"]
Sep 10 13:49:23.462593 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.462556    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"]
Sep 10 13:49:23.462786 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.462760    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k" podUID="ce6b50d9-3848-4c8e-aafc-099df8986d82" containerName="payload-processing" containerID="cri-o://228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550" gracePeriod=30
Sep 10 13:49:23.480018 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.479992    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"]
Sep 10 13:49:23.480248 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.480223    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6" podUID="f116b39c-ce19-4577-b14c-36b4ce47a41b" containerName="payload-pre-processing" containerID="cri-o://fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e" gracePeriod=30
Sep 10 13:49:23.678011 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.676014    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb697720-d6c8-45db-be71-2635a40605ba" path="/var/lib/kubelet/pods/bb697720-d6c8-45db-be71-2635a40605ba/volumes"
Sep 10 13:49:23.701909 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.701878    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"]
Sep 10 13:49:23.704161 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.704137    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e6d037-44s6m"]
Sep 10 13:49:23.744952 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.744932    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:49:23.747787 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.747767    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:49:23.821134 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.821106    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjzb8\" (UniqueName: \"kubernetes.io/projected/ce6b50d9-3848-4c8e-aafc-099df8986d82-kube-api-access-mjzb8\") pod \"ce6b50d9-3848-4c8e-aafc-099df8986d82\" (UID: \"ce6b50d9-3848-4c8e-aafc-099df8986d82\") "
Sep 10 13:49:23.821385 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.821187    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f116b39c-ce19-4577-b14c-36b4ce47a41b-plugins-config-volume\") pod \"f116b39c-ce19-4577-b14c-36b4ce47a41b\" (UID: \"f116b39c-ce19-4577-b14c-36b4ce47a41b\") "
Sep 10 13:49:23.821385 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.821230    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce6b50d9-3848-4c8e-aafc-099df8986d82-plugins-config-volume\") pod \"ce6b50d9-3848-4c8e-aafc-099df8986d82\" (UID: \"ce6b50d9-3848-4c8e-aafc-099df8986d82\") "
Sep 10 13:49:23.821385 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.821278    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grnkv\" (UniqueName: \"kubernetes.io/projected/f116b39c-ce19-4577-b14c-36b4ce47a41b-kube-api-access-grnkv\") pod \"f116b39c-ce19-4577-b14c-36b4ce47a41b\" (UID: \"f116b39c-ce19-4577-b14c-36b4ce47a41b\") "
Sep 10 13:49:23.821629 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.821601    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f116b39c-ce19-4577-b14c-36b4ce47a41b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f116b39c-ce19-4577-b14c-36b4ce47a41b" (UID: "f116b39c-ce19-4577-b14c-36b4ce47a41b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:49:23.821629 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.821618    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce6b50d9-3848-4c8e-aafc-099df8986d82-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce6b50d9-3848-4c8e-aafc-099df8986d82" (UID: "ce6b50d9-3848-4c8e-aafc-099df8986d82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:49:23.823137 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.823117    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce6b50d9-3848-4c8e-aafc-099df8986d82-kube-api-access-mjzb8" (OuterVolumeSpecName: "kube-api-access-mjzb8") pod "ce6b50d9-3848-4c8e-aafc-099df8986d82" (UID: "ce6b50d9-3848-4c8e-aafc-099df8986d82"). InnerVolumeSpecName "kube-api-access-mjzb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:49:23.823364 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.823340    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f116b39c-ce19-4577-b14c-36b4ce47a41b-kube-api-access-grnkv" (OuterVolumeSpecName: "kube-api-access-grnkv") pod "f116b39c-ce19-4577-b14c-36b4ce47a41b" (UID: "f116b39c-ce19-4577-b14c-36b4ce47a41b"). InnerVolumeSpecName "kube-api-access-grnkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:49:23.922697 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.922653    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grnkv\" (UniqueName: \"kubernetes.io/projected/f116b39c-ce19-4577-b14c-36b4ce47a41b-kube-api-access-grnkv\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:23.922697 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.922692    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjzb8\" (UniqueName: \"kubernetes.io/projected/ce6b50d9-3848-4c8e-aafc-099df8986d82-kube-api-access-mjzb8\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:23.922697 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.922701    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f116b39c-ce19-4577-b14c-36b4ce47a41b-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:23.922697 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:23.922710    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce6b50d9-3848-4c8e-aafc-099df8986d82-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:49:24.096037 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.095933    2549 generic.go:356] "Generic (PLEG): container finished" podID="ce6b50d9-3848-4c8e-aafc-099df8986d82" containerID="228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550" exitCode=0
Sep 10 13:49:24.096037 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.095980    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"
Sep 10 13:49:24.096037 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.096015    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k" event={"ID":"ce6b50d9-3848-4c8e-aafc-099df8986d82","Type":"ContainerDied","Data":"228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550"}
Sep 10 13:49:24.096327 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.096067    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k" event={"ID":"ce6b50d9-3848-4c8e-aafc-099df8986d82","Type":"ContainerDied","Data":"f312833d5470d49c7ed889d9482decec1b26475d6a80bbd96268926e319c7833"}
Sep 10 13:49:24.096327 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.096095    2549 scope.go:117] "RemoveContainer" containerID="228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550"
Sep 10 13:49:24.097513 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.097422    2549 generic.go:356] "Generic (PLEG): container finished" podID="f116b39c-ce19-4577-b14c-36b4ce47a41b" containerID="fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e" exitCode=0
Sep 10 13:49:24.097513 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.097446    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6" event={"ID":"f116b39c-ce19-4577-b14c-36b4ce47a41b","Type":"ContainerDied","Data":"fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e"}
Sep 10 13:49:24.097513 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.097486    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"
Sep 10 13:49:24.097757 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.097491    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6" event={"ID":"f116b39c-ce19-4577-b14c-36b4ce47a41b","Type":"ContainerDied","Data":"072c644f55b1dbbafe7651a07bae2f59fe9ca20be83f5b0d06f8caad0b9c8276"}
Sep 10 13:49:24.105280 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.105139    2549 scope.go:117] "RemoveContainer" containerID="228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550"
Sep 10 13:49:24.105489 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:49:24.105451    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550\": container with ID starting with 228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550 not found: ID does not exist" containerID="228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550"
Sep 10 13:49:24.105555 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.105488    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550"} err="failed to get container status \"228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550\": rpc error: code = NotFound desc = could not find container \"228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550\": container with ID starting with 228b19890796bd08e6586cfd39b722da5bfe07ce703db4ca51b9a150543da550 not found: ID does not exist"
Sep 10 13:49:24.105555 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.105512    2549 scope.go:117] "RemoveContainer" containerID="fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e"
Sep 10 13:49:24.113445 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.113431    2549 scope.go:117] "RemoveContainer" containerID="fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e"
Sep 10 13:49:24.113642 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:49:24.113624    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e\": container with ID starting with fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e not found: ID does not exist" containerID="fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e"
Sep 10 13:49:24.113711 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.113647    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e"} err="failed to get container status \"fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e\": rpc error: code = NotFound desc = could not find container \"fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e\": container with ID starting with fe8e75d31427c5918715f083f53c5b89b9c096e469e8d3b5b727953e4fb3c03e not found: ID does not exist"
Sep 10 13:49:24.128028 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:49:24.127999    2549 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6b50d9_3848_4c8e_aafc_099df8986d82.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6b50d9_3848_4c8e_aafc_099df8986d82.slice/crio-f312833d5470d49c7ed889d9482decec1b26475d6a80bbd96268926e319c7833\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf116b39c_ce19_4577_b14c_36b4ce47a41b.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:49:24.130743 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.130724    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"]
Sep 10 13:49:24.136233 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.136215    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e6d037-57cfd6cff8-q6bb6"]
Sep 10 13:49:24.152516 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.152500    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"]
Sep 10 13:49:24.155018 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:24.155001    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e6d037-7c4b8f8fc7-v4d2k"]
Sep 10 13:49:25.673776 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:25.673735    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f3b8613-2548-45f8-9b8f-033261e72e3e" path="/var/lib/kubelet/pods/4f3b8613-2548-45f8-9b8f-033261e72e3e/volumes"
Sep 10 13:49:25.674267 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:25.674244    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce6b50d9-3848-4c8e-aafc-099df8986d82" path="/var/lib/kubelet/pods/ce6b50d9-3848-4c8e-aafc-099df8986d82/volumes"
Sep 10 13:49:25.674601 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:25.674573    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f116b39c-ce19-4577-b14c-36b4ce47a41b" path="/var/lib/kubelet/pods/f116b39c-ce19-4577-b14c-36b4ce47a41b/volumes"
Sep 10 13:49:33.091789 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:33.091752    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:49:39.427949 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.427918    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"]
Sep 10 13:49:39.428295 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.428194    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f116b39c-ce19-4577-b14c-36b4ce47a41b" containerName="payload-pre-processing"
Sep 10 13:49:39.428295 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.428205    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="f116b39c-ce19-4577-b14c-36b4ce47a41b" containerName="payload-pre-processing"
Sep 10 13:49:39.428295 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.428222    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce6b50d9-3848-4c8e-aafc-099df8986d82" containerName="payload-processing"
Sep 10 13:49:39.428295 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.428227    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce6b50d9-3848-4c8e-aafc-099df8986d82" containerName="payload-processing"
Sep 10 13:49:39.428295 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.428269    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="f116b39c-ce19-4577-b14c-36b4ce47a41b" containerName="payload-pre-processing"
Sep 10 13:49:39.428295 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.428277    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce6b50d9-3848-4c8e-aafc-099df8986d82" containerName="payload-processing"
Sep 10 13:49:39.432291 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.432273    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 13:49:39.434671 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.434652    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-579ae1-dockercfg-s8sjp\""
Sep 10 13:49:39.435551 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.435534    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-579ae1\""
Sep 10 13:49:39.444526 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.444503    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"]
Sep 10 13:49:39.487310 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.487282    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"]
Sep 10 13:49:39.491152 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.491134    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 13:49:39.510741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.510717    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"]
Sep 10 13:49:39.548590 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.548562    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56\" (UID: \"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 13:49:39.548705 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.548604    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72bq7\" (UniqueName: \"kubernetes.io/projected/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-kube-api-access-72bq7\") pod \"payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56\" (UID: \"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 13:49:39.548705 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.548626    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnf55\" (UniqueName: \"kubernetes.io/projected/5859e036-3956-418a-9af2-9df64e00dbf9-kube-api-access-lnf55\") pod \"payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96\" (UID: \"5859e036-3956-418a-9af2-9df64e00dbf9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 13:49:39.548705 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.548652    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5859e036-3956-418a-9af2-9df64e00dbf9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96\" (UID: \"5859e036-3956-418a-9af2-9df64e00dbf9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 13:49:39.649624 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.649597    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56\" (UID: \"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 13:49:39.649764 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.649637    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72bq7\" (UniqueName: \"kubernetes.io/projected/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-kube-api-access-72bq7\") pod \"payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56\" (UID: \"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 13:49:39.649764 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.649666    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnf55\" (UniqueName: \"kubernetes.io/projected/5859e036-3956-418a-9af2-9df64e00dbf9-kube-api-access-lnf55\") pod \"payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96\" (UID: \"5859e036-3956-418a-9af2-9df64e00dbf9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 13:49:39.649935 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.649794    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5859e036-3956-418a-9af2-9df64e00dbf9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96\" (UID: \"5859e036-3956-418a-9af2-9df64e00dbf9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 13:49:39.650341 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.650323    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56\" (UID: \"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 13:49:39.650663 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.650641    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5859e036-3956-418a-9af2-9df64e00dbf9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96\" (UID: \"5859e036-3956-418a-9af2-9df64e00dbf9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 13:49:39.675315 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.675285    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnf55\" (UniqueName: \"kubernetes.io/projected/5859e036-3956-418a-9af2-9df64e00dbf9-kube-api-access-lnf55\") pod \"payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96\" (UID: \"5859e036-3956-418a-9af2-9df64e00dbf9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 13:49:39.676793 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.676770    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72bq7\" (UniqueName: \"kubernetes.io/projected/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-kube-api-access-72bq7\") pod \"payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56\" (UID: \"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 13:49:39.742093 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.742023    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 13:49:39.800504 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:39.800268    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 13:49:40.023119 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:40.023048    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"]
Sep 10 13:49:40.025820 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:49:40.025792    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5859e036_3956_418a_9af2_9df64e00dbf9.slice/crio-8dbbb1df30c8df2d4c7c1c43309ee40df7dd1202c697109915604a930f0d675e WatchSource:0}: Error finding container 8dbbb1df30c8df2d4c7c1c43309ee40df7dd1202c697109915604a930f0d675e: Status 404 returned error can't find the container with id 8dbbb1df30c8df2d4c7c1c43309ee40df7dd1202c697109915604a930f0d675e
Sep 10 13:49:40.082663 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:40.082634    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"]
Sep 10 13:49:40.085120 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:49:40.085098    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5189b16f_1b97_4dbf_939f_41e8a4dc3f6d.slice/crio-eeb886f626dbc5939ddf432f1bbe162f859f134a3ef39b757804be1c03a7822e WatchSource:0}: Error finding container eeb886f626dbc5939ddf432f1bbe162f859f134a3ef39b757804be1c03a7822e: Status 404 returned error can't find the container with id eeb886f626dbc5939ddf432f1bbe162f859f134a3ef39b757804be1c03a7822e
Sep 10 13:49:40.173255 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:40.173213    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56" event={"ID":"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d","Type":"ContainerStarted","Data":"7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d"}
Sep 10 13:49:40.173445 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:40.173301    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56" event={"ID":"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d","Type":"ContainerStarted","Data":"eeb886f626dbc5939ddf432f1bbe162f859f134a3ef39b757804be1c03a7822e"}
Sep 10 13:49:40.173445 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:40.173330    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 13:49:40.174983 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:40.174953    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96" event={"ID":"5859e036-3956-418a-9af2-9df64e00dbf9","Type":"ContainerStarted","Data":"ecd2377bda2aa3fd6cd46fd7a0be96586eb5a8367755514f83ee9c74d1cf8a21"}
Sep 10 13:49:40.175098 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:40.174997    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96" event={"ID":"5859e036-3956-418a-9af2-9df64e00dbf9","Type":"ContainerStarted","Data":"8dbbb1df30c8df2d4c7c1c43309ee40df7dd1202c697109915604a930f0d675e"}
Sep 10 13:49:40.175098 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:40.175057    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 13:49:40.214555 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:40.214489    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56" podStartSLOduration=1.214471233 podStartE2EDuration="1.214471233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:49:40.211338902 +0000 UTC m=+1139.117896933" watchObservedRunningTime="2026-09-10 13:49:40.214471233 +0000 UTC m=+1139.121029264"
Sep 10 13:49:40.270206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:40.270153    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96" podStartSLOduration=1.270137579 podStartE2EDuration="1.270137579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:49:40.268385695 +0000 UTC m=+1139.174943726" watchObservedRunningTime="2026-09-10 13:49:40.270137579 +0000 UTC m=+1139.176695609"
Sep 10 13:49:46.114962 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.114926    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"]
Sep 10 13:49:46.119717 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.119698    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:49:46.122686 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.122663    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-b3723e\""
Sep 10 13:49:46.123267 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.123249    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-b3723e-dockercfg-vgdbf\""
Sep 10 13:49:46.136405 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.136382    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"]
Sep 10 13:49:46.203581 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.203547    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzgbr\" (UniqueName: \"kubernetes.io/projected/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-kube-api-access-qzgbr\") pod \"payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j\" (UID: \"9e7dfc60-9855-4211-9c3e-155cb43ae2ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:49:46.203743 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.203626    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j\" (UID: \"9e7dfc60-9855-4211-9c3e-155cb43ae2ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:49:46.226127 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.226100    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"]
Sep 10 13:49:46.229325 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.229309    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:49:46.265848 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.265822    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"]
Sep 10 13:49:46.304239 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.304211    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j\" (UID: \"9e7dfc60-9855-4211-9c3e-155cb43ae2ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:49:46.304397 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.304263    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tt4d\" (UniqueName: \"kubernetes.io/projected/7e13f416-6911-4b1a-9966-dc0747c19b59-kube-api-access-2tt4d\") pod \"payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x\" (UID: \"7e13f416-6911-4b1a-9966-dc0747c19b59\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:49:46.304397 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.304304    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzgbr\" (UniqueName: \"kubernetes.io/projected/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-kube-api-access-qzgbr\") pod \"payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j\" (UID: \"9e7dfc60-9855-4211-9c3e-155cb43ae2ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:49:46.304397 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.304327    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e13f416-6911-4b1a-9966-dc0747c19b59-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x\" (UID: \"7e13f416-6911-4b1a-9966-dc0747c19b59\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:49:46.305026 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.304998    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j\" (UID: \"9e7dfc60-9855-4211-9c3e-155cb43ae2ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:49:46.324247 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.324220    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzgbr\" (UniqueName: \"kubernetes.io/projected/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-kube-api-access-qzgbr\") pod \"payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j\" (UID: \"9e7dfc60-9855-4211-9c3e-155cb43ae2ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:49:46.404917 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.404829    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tt4d\" (UniqueName: \"kubernetes.io/projected/7e13f416-6911-4b1a-9966-dc0747c19b59-kube-api-access-2tt4d\") pod \"payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x\" (UID: \"7e13f416-6911-4b1a-9966-dc0747c19b59\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:49:46.404917 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.404888    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e13f416-6911-4b1a-9966-dc0747c19b59-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x\" (UID: \"7e13f416-6911-4b1a-9966-dc0747c19b59\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:49:46.405502 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.405483    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e13f416-6911-4b1a-9966-dc0747c19b59-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x\" (UID: \"7e13f416-6911-4b1a-9966-dc0747c19b59\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:49:46.417294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.417272    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tt4d\" (UniqueName: \"kubernetes.io/projected/7e13f416-6911-4b1a-9966-dc0747c19b59-kube-api-access-2tt4d\") pod \"payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x\" (UID: \"7e13f416-6911-4b1a-9966-dc0747c19b59\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:49:46.428498 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.428479    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:49:46.538414 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.538385    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:49:46.705148 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.705122    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"]
Sep 10 13:49:46.707167 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:49:46.707146    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e7dfc60_9855_4211_9c3e_155cb43ae2ac.slice/crio-864f562b32900fe0ce8644df14512ecba74dca4c0e305af5e39cb3c0996632ae WatchSource:0}: Error finding container 864f562b32900fe0ce8644df14512ecba74dca4c0e305af5e39cb3c0996632ae: Status 404 returned error can't find the container with id 864f562b32900fe0ce8644df14512ecba74dca4c0e305af5e39cb3c0996632ae
Sep 10 13:49:46.819580 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:46.819549    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"]
Sep 10 13:49:46.822077 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:49:46.822056    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e13f416_6911_4b1a_9966_dc0747c19b59.slice/crio-8420ee974ccea15b333d079d094d8fb507f4f6f759ff94fccba66fb7432d3821 WatchSource:0}: Error finding container 8420ee974ccea15b333d079d094d8fb507f4f6f759ff94fccba66fb7432d3821: Status 404 returned error can't find the container with id 8420ee974ccea15b333d079d094d8fb507f4f6f759ff94fccba66fb7432d3821
Sep 10 13:49:47.206943 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:47.206902    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x" event={"ID":"7e13f416-6911-4b1a-9966-dc0747c19b59","Type":"ContainerStarted","Data":"da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745"}
Sep 10 13:49:47.207386 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:47.206965    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x" event={"ID":"7e13f416-6911-4b1a-9966-dc0747c19b59","Type":"ContainerStarted","Data":"8420ee974ccea15b333d079d094d8fb507f4f6f759ff94fccba66fb7432d3821"}
Sep 10 13:49:47.207386 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:47.206993    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:49:47.208569 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:47.208532    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j" event={"ID":"9e7dfc60-9855-4211-9c3e-155cb43ae2ac","Type":"ContainerStarted","Data":"f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820"}
Sep 10 13:49:47.208701 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:47.208582    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j" event={"ID":"9e7dfc60-9855-4211-9c3e-155cb43ae2ac","Type":"ContainerStarted","Data":"864f562b32900fe0ce8644df14512ecba74dca4c0e305af5e39cb3c0996632ae"}
Sep 10 13:49:47.208701 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:47.208665    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:49:47.239786 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:47.239747    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x" podStartSLOduration=1.2397339920000001 podStartE2EDuration="1.239733992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:49:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:49:47.23944419 +0000 UTC m=+1146.146002239" watchObservedRunningTime="2026-09-10 13:49:47.239733992 +0000 UTC m=+1146.146292022"
Sep 10 13:49:47.284495 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:47.284452    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j" podStartSLOduration=1.284440361 podStartE2EDuration="1.284440361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:49:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:49:47.279077562 +0000 UTC m=+1146.185635591" watchObservedRunningTime="2026-09-10 13:49:47.284440361 +0000 UTC m=+1146.190998389"
Sep 10 13:49:51.181440 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:51.181405    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 13:49:51.181822 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:51.181476    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 13:49:58.214687 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:58.214648    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:49:58.215090 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:49:58.214729    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:50:08.892145 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:08.892072    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7841-f81539"]
Sep 10 13:50:08.896237 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:08.896222    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-f81539"
Sep 10 13:50:08.898602 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:08.898574    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-g8rj6\""
Sep 10 13:50:08.904552 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:08.904527    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7841-f81539"]
Sep 10 13:50:08.991669 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:08.991631    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv774\" (UniqueName: \"kubernetes.io/projected/c6c2303c-4d47-41de-88a1-f948ad1f714a-kube-api-access-mv774\") pod \"test-curl-7841-f81539\" (UID: \"c6c2303c-4d47-41de-88a1-f948ad1f714a\") " pod="opendatahub/test-curl-7841-f81539"
Sep 10 13:50:09.092218 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:09.092186    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv774\" (UniqueName: \"kubernetes.io/projected/c6c2303c-4d47-41de-88a1-f948ad1f714a-kube-api-access-mv774\") pod \"test-curl-7841-f81539\" (UID: \"c6c2303c-4d47-41de-88a1-f948ad1f714a\") " pod="opendatahub/test-curl-7841-f81539"
Sep 10 13:50:09.101737 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:09.101712    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv774\" (UniqueName: \"kubernetes.io/projected/c6c2303c-4d47-41de-88a1-f948ad1f714a-kube-api-access-mv774\") pod \"test-curl-7841-f81539\" (UID: \"c6c2303c-4d47-41de-88a1-f948ad1f714a\") " pod="opendatahub/test-curl-7841-f81539"
Sep 10 13:50:09.205343 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:09.205316    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-f81539"
Sep 10 13:50:09.479636 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:09.479605    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7841-f81539"]
Sep 10 13:50:09.482174 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:50:09.482143    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6c2303c_4d47_41de_88a1_f948ad1f714a.slice/crio-c8d866053697e080141ebafcaf77628f921cfe6b74d70f85e78d8b93b4489c2c WatchSource:0}: Error finding container c8d866053697e080141ebafcaf77628f921cfe6b74d70f85e78d8b93b4489c2c: Status 404 returned error can't find the container with id c8d866053697e080141ebafcaf77628f921cfe6b74d70f85e78d8b93b4489c2c
Sep 10 13:50:10.308942 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:10.308900    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7841-f81539" event={"ID":"c6c2303c-4d47-41de-88a1-f948ad1f714a","Type":"ContainerStarted","Data":"9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649"}
Sep 10 13:50:10.308942 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:10.308947    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7841-f81539" event={"ID":"c6c2303c-4d47-41de-88a1-f948ad1f714a","Type":"ContainerStarted","Data":"c8d866053697e080141ebafcaf77628f921cfe6b74d70f85e78d8b93b4489c2c"}
Sep 10 13:50:10.330677 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:10.330631    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7841-f81539" podStartSLOduration=2.330616491 podStartE2EDuration="2.330616491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:50:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:50:10.329684983 +0000 UTC m=+1169.236243014" watchObservedRunningTime="2026-09-10 13:50:10.330616491 +0000 UTC m=+1169.237174520"
Sep 10 13:50:11.165677 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.165643    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7841-f81539"]
Sep 10 13:50:11.170626 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.170597    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7841-f81539"]
Sep 10 13:50:11.302471 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.302390    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7841-ca3a43"]
Sep 10 13:50:11.302714 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.302701    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6c2303c-4d47-41de-88a1-f948ad1f714a" containerName="test-curl-7841-f81539"
Sep 10 13:50:11.302767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.302715    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6c2303c-4d47-41de-88a1-f948ad1f714a" containerName="test-curl-7841-f81539"
Sep 10 13:50:11.302806 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.302782    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6c2303c-4d47-41de-88a1-f948ad1f714a" containerName="test-curl-7841-f81539"
Sep 10 13:50:11.305501 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.305483    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-ca3a43"
Sep 10 13:50:11.313287 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:11.313260    2549 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7841-f81539" container="test-curl-7841-f81539"
Sep 10 13:50:11.315218 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.315193    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7841-ca3a43"]
Sep 10 13:50:11.409021 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.408993    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxnlc\" (UniqueName: \"kubernetes.io/projected/be228301-46a1-418b-8fec-ed796897fd9f-kube-api-access-bxnlc\") pod \"test-curl-7841-ca3a43\" (UID: \"be228301-46a1-418b-8fec-ed796897fd9f\") " pod="opendatahub/test-curl-7841-ca3a43"
Sep 10 13:50:11.510171 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.510136    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxnlc\" (UniqueName: \"kubernetes.io/projected/be228301-46a1-418b-8fec-ed796897fd9f-kube-api-access-bxnlc\") pod \"test-curl-7841-ca3a43\" (UID: \"be228301-46a1-418b-8fec-ed796897fd9f\") " pod="opendatahub/test-curl-7841-ca3a43"
Sep 10 13:50:11.525325 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.525294    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxnlc\" (UniqueName: \"kubernetes.io/projected/be228301-46a1-418b-8fec-ed796897fd9f-kube-api-access-bxnlc\") pod \"test-curl-7841-ca3a43\" (UID: \"be228301-46a1-418b-8fec-ed796897fd9f\") " pod="opendatahub/test-curl-7841-ca3a43"
Sep 10 13:50:11.615001 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.614927    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-ca3a43"
Sep 10 13:50:11.906068 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:11.905994    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7841-ca3a43"]
Sep 10 13:50:11.909003 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:50:11.908978    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe228301_46a1_418b_8fec_ed796897fd9f.slice/crio-136dfe985cf8e473ff4c77b2c50772957279be5f0961a1b228f41c760e280405 WatchSource:0}: Error finding container 136dfe985cf8e473ff4c77b2c50772957279be5f0961a1b228f41c760e280405: Status 404 returned error can't find the container with id 136dfe985cf8e473ff4c77b2c50772957279be5f0961a1b228f41c760e280405
Sep 10 13:50:12.318919 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:12.318882    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7841-ca3a43" event={"ID":"be228301-46a1-418b-8fec-ed796897fd9f","Type":"ContainerStarted","Data":"6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded"}
Sep 10 13:50:12.319398 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:12.318927    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7841-ca3a43" event={"ID":"be228301-46a1-418b-8fec-ed796897fd9f","Type":"ContainerStarted","Data":"136dfe985cf8e473ff4c77b2c50772957279be5f0961a1b228f41c760e280405"}
Sep 10 13:50:12.319398 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:12.319054    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7841-f81539" podUID="c6c2303c-4d47-41de-88a1-f948ad1f714a" containerName="test-curl-7841-f81539" containerID="cri-o://9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649" gracePeriod=2
Sep 10 13:50:13.119251 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.119143    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7841-ca3a43" podStartSLOduration=2.119125169 podStartE2EDuration="2.119125169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:50:12.363980391 +0000 UTC m=+1171.270538420" watchObservedRunningTime="2026-09-10 13:50:13.119125169 +0000 UTC m=+1172.025683198"
Sep 10 13:50:13.120053 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.120036    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7841-ca3a43"]
Sep 10 13:50:13.141423 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.141396    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7841-ca3a43"]
Sep 10 13:50:13.282716 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.282687    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7841-31c79b"]
Sep 10 13:50:13.283117 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.283102    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be228301-46a1-418b-8fec-ed796897fd9f" containerName="test-curl-7841-ca3a43"
Sep 10 13:50:13.283167 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.283120    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="be228301-46a1-418b-8fec-ed796897fd9f" containerName="test-curl-7841-ca3a43"
Sep 10 13:50:13.283247 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.283181    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="be228301-46a1-418b-8fec-ed796897fd9f" containerName="test-curl-7841-ca3a43"
Sep 10 13:50:13.286196 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.286181    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-31c79b"
Sep 10 13:50:13.290055 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:13.290020    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7841-31c79b\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="35c78244-b747-4b89-a4fd-e404b49c8da9" pod="opendatahub/test-curl-7841-31c79b"
Sep 10 13:50:13.317136 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.317109    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7841-31c79b"]
Sep 10 13:50:13.324341 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:13.324315    2549 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7841-ca3a43" container="test-curl-7841-ca3a43"
Sep 10 13:50:13.332876 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.332841    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6555t\" (UniqueName: \"kubernetes.io/projected/35c78244-b747-4b89-a4fd-e404b49c8da9-kube-api-access-6555t\") pod \"test-curl-7841-31c79b\" (UID: \"35c78244-b747-4b89-a4fd-e404b49c8da9\") " pod="opendatahub/test-curl-7841-31c79b"
Sep 10 13:50:13.433498 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.433471    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6555t\" (UniqueName: \"kubernetes.io/projected/35c78244-b747-4b89-a4fd-e404b49c8da9-kube-api-access-6555t\") pod \"test-curl-7841-31c79b\" (UID: \"35c78244-b747-4b89-a4fd-e404b49c8da9\") " pod="opendatahub/test-curl-7841-31c79b"
Sep 10 13:50:13.444878 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.444848    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6555t\" (UniqueName: \"kubernetes.io/projected/35c78244-b747-4b89-a4fd-e404b49c8da9-kube-api-access-6555t\") pod \"test-curl-7841-31c79b\" (UID: \"35c78244-b747-4b89-a4fd-e404b49c8da9\") " pod="opendatahub/test-curl-7841-31c79b"
Sep 10 13:50:13.540436 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.540408    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"]
Sep 10 13:50:13.543730 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.543709    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 13:50:13.548413 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.548392    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-d5c012\""
Sep 10 13:50:13.548686 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.548672    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-d5c012-dockercfg-d2c9c\""
Sep 10 13:50:13.576835 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.576813    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"]
Sep 10 13:50:13.594810 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.594786    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-31c79b"
Sep 10 13:50:13.636015 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.635235    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px69x\" (UniqueName: \"kubernetes.io/projected/4980fb51-e869-4632-93e1-bb80c85660da-kube-api-access-px69x\") pod \"payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259\" (UID: \"4980fb51-e869-4632-93e1-bb80c85660da\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 13:50:13.636015 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.635321    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4980fb51-e869-4632-93e1-bb80c85660da-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259\" (UID: \"4980fb51-e869-4632-93e1-bb80c85660da\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 13:50:13.736525 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.736441    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px69x\" (UniqueName: \"kubernetes.io/projected/4980fb51-e869-4632-93e1-bb80c85660da-kube-api-access-px69x\") pod \"payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259\" (UID: \"4980fb51-e869-4632-93e1-bb80c85660da\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 13:50:13.736688 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.736647    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4980fb51-e869-4632-93e1-bb80c85660da-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259\" (UID: \"4980fb51-e869-4632-93e1-bb80c85660da\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 13:50:13.737356 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.737332    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4980fb51-e869-4632-93e1-bb80c85660da-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259\" (UID: \"4980fb51-e869-4632-93e1-bb80c85660da\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 13:50:13.748001 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.747979    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px69x\" (UniqueName: \"kubernetes.io/projected/4980fb51-e869-4632-93e1-bb80c85660da-kube-api-access-px69x\") pod \"payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259\" (UID: \"4980fb51-e869-4632-93e1-bb80c85660da\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 13:50:13.854001 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.853969    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 13:50:13.879100 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:13.879072    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7841-31c79b"]
Sep 10 13:50:13.882065 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:50:13.882041    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35c78244_b747_4b89_a4fd_e404b49c8da9.slice/crio-a97e524a41615655d39f602f52eb9a7cf6088e690838181567c24d9f57f9ad98 WatchSource:0}: Error finding container a97e524a41615655d39f602f52eb9a7cf6088e690838181567c24d9f57f9ad98: Status 404 returned error can't find the container with id a97e524a41615655d39f602f52eb9a7cf6088e690838181567c24d9f57f9ad98
Sep 10 13:50:14.144842 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.144765    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"]
Sep 10 13:50:14.148388 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:50:14.148361    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4980fb51_e869_4632_93e1_bb80c85660da.slice/crio-4429e8e40d055097c62d0bed9100cbed7ad1e17e66ec615aca0c47ec7d4ac6ef WatchSource:0}: Error finding container 4429e8e40d055097c62d0bed9100cbed7ad1e17e66ec615aca0c47ec7d4ac6ef: Status 404 returned error can't find the container with id 4429e8e40d055097c62d0bed9100cbed7ad1e17e66ec615aca0c47ec7d4ac6ef
Sep 10 13:50:14.330301 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.330271    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7841-31c79b" event={"ID":"35c78244-b747-4b89-a4fd-e404b49c8da9","Type":"ContainerStarted","Data":"6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1"}
Sep 10 13:50:14.330656 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.330317    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7841-31c79b" event={"ID":"35c78244-b747-4b89-a4fd-e404b49c8da9","Type":"ContainerStarted","Data":"a97e524a41615655d39f602f52eb9a7cf6088e690838181567c24d9f57f9ad98"}
Sep 10 13:50:14.331827 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.331804    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259" event={"ID":"4980fb51-e869-4632-93e1-bb80c85660da","Type":"ContainerStarted","Data":"e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec"}
Sep 10 13:50:14.331955 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.331846    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259" event={"ID":"4980fb51-e869-4632-93e1-bb80c85660da","Type":"ContainerStarted","Data":"4429e8e40d055097c62d0bed9100cbed7ad1e17e66ec615aca0c47ec7d4ac6ef"}
Sep 10 13:50:14.331955 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.331858    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7841-ca3a43" podUID="be228301-46a1-418b-8fec-ed796897fd9f" containerName="test-curl-7841-ca3a43" containerID="cri-o://6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded" gracePeriod=2
Sep 10 13:50:14.331955 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.331896    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 13:50:14.372466 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.372398    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7841-31c79b" podStartSLOduration=1.37237539 podStartE2EDuration="1.37237539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:50:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:50:14.369403536 +0000 UTC m=+1173.275961565" watchObservedRunningTime="2026-09-10 13:50:14.37237539 +0000 UTC m=+1173.278933421"
Sep 10 13:50:14.410378 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.410317    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259" podStartSLOduration=1.410297705 podStartE2EDuration="1.410297705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:50:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:50:14.409892943 +0000 UTC m=+1173.316450996" watchObservedRunningTime="2026-09-10 13:50:14.410297705 +0000 UTC m=+1173.316855737"
Sep 10 13:50:14.511236 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.511208    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-f81539"
Sep 10 13:50:14.543304 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.543278    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv774\" (UniqueName: \"kubernetes.io/projected/c6c2303c-4d47-41de-88a1-f948ad1f714a-kube-api-access-mv774\") pod \"c6c2303c-4d47-41de-88a1-f948ad1f714a\" (UID: \"c6c2303c-4d47-41de-88a1-f948ad1f714a\") "
Sep 10 13:50:14.545262 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.545236    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c2303c-4d47-41de-88a1-f948ad1f714a-kube-api-access-mv774" (OuterVolumeSpecName: "kube-api-access-mv774") pod "c6c2303c-4d47-41de-88a1-f948ad1f714a" (UID: "c6c2303c-4d47-41de-88a1-f948ad1f714a"). InnerVolumeSpecName "kube-api-access-mv774". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:50:14.644536 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:14.644507    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv774\" (UniqueName: \"kubernetes.io/projected/c6c2303c-4d47-41de-88a1-f948ad1f714a-kube-api-access-mv774\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:50:15.081567 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.081537    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7841-31c79b"]
Sep 10 13:50:15.090142 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.090113    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7841-31c79b"]
Sep 10 13:50:15.218092 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.218059    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7841-88c29c"]
Sep 10 13:50:15.218458 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.218441    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c78244-b747-4b89-a4fd-e404b49c8da9" containerName="test-curl-7841-31c79b"
Sep 10 13:50:15.218552 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.218461    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c78244-b747-4b89-a4fd-e404b49c8da9" containerName="test-curl-7841-31c79b"
Sep 10 13:50:15.218618 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.218581    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="35c78244-b747-4b89-a4fd-e404b49c8da9" containerName="test-curl-7841-31c79b"
Sep 10 13:50:15.221722 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.221702    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-88c29c"
Sep 10 13:50:15.236266 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.236245    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7841-88c29c"]
Sep 10 13:50:15.251273 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.251247    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv557\" (UniqueName: \"kubernetes.io/projected/779f1ba9-28b6-4d99-bf5c-aa15efaf5db5-kube-api-access-pv557\") pod \"test-curl-7841-88c29c\" (UID: \"779f1ba9-28b6-4d99-bf5c-aa15efaf5db5\") " pod="opendatahub/test-curl-7841-88c29c"
Sep 10 13:50:15.337726 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.337695    2549 generic.go:356] "Generic (PLEG): container finished" podID="c6c2303c-4d47-41de-88a1-f948ad1f714a" containerID="9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649" exitCode=137
Sep 10 13:50:15.338180 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.337752    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-f81539"
Sep 10 13:50:15.338250 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:15.338182    2549 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7841-31c79b" container="test-curl-7841-31c79b"
Sep 10 13:50:15.338625 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.337765    2549 scope.go:117] "RemoveContainer" containerID="9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649"
Sep 10 13:50:15.346875 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.346850    2549 scope.go:117] "RemoveContainer" containerID="9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649"
Sep 10 13:50:15.347157 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:15.347126    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649\": container with ID starting with 9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649 not found: ID does not exist" containerID="9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649"
Sep 10 13:50:15.347204 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.347164    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649"} err="failed to get container status \"9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649\": rpc error: code = NotFound desc = could not find container \"9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649\": container with ID starting with 9a890044e2a01c30557cef918d76229657fbfae53fba799a822adf1276d77649 not found: ID does not exist"
Sep 10 13:50:15.348239 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:15.348221    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7841-f81539\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="c6c2303c-4d47-41de-88a1-f948ad1f714a" pod="opendatahub/test-curl-7841-f81539"
Sep 10 13:50:15.352546 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.352530    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv557\" (UniqueName: \"kubernetes.io/projected/779f1ba9-28b6-4d99-bf5c-aa15efaf5db5-kube-api-access-pv557\") pod \"test-curl-7841-88c29c\" (UID: \"779f1ba9-28b6-4d99-bf5c-aa15efaf5db5\") " pod="opendatahub/test-curl-7841-88c29c"
Sep 10 13:50:15.366145 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.366123    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv557\" (UniqueName: \"kubernetes.io/projected/779f1ba9-28b6-4d99-bf5c-aa15efaf5db5-kube-api-access-pv557\") pod \"test-curl-7841-88c29c\" (UID: \"779f1ba9-28b6-4d99-bf5c-aa15efaf5db5\") " pod="opendatahub/test-curl-7841-88c29c"
Sep 10 13:50:15.534311 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.534229    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-88c29c"
Sep 10 13:50:15.673789 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.673759    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6c2303c-4d47-41de-88a1-f948ad1f714a" path="/var/lib/kubelet/pods/c6c2303c-4d47-41de-88a1-f948ad1f714a/volumes"
Sep 10 13:50:15.805220 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:15.805152    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7841-88c29c"]
Sep 10 13:50:15.808633 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:50:15.808605    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod779f1ba9_28b6_4d99_bf5c_aa15efaf5db5.slice/crio-b6a97d5f38e86cb9372281e235f798ad9c19b59590fc1a85c2fcc26da311381d WatchSource:0}: Error finding container b6a97d5f38e86cb9372281e235f798ad9c19b59590fc1a85c2fcc26da311381d: Status 404 returned error can't find the container with id b6a97d5f38e86cb9372281e235f798ad9c19b59590fc1a85c2fcc26da311381d
Sep 10 13:50:16.344987 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:16.344961    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7841-88c29c" event={"ID":"779f1ba9-28b6-4d99-bf5c-aa15efaf5db5","Type":"ContainerStarted","Data":"2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec"}
Sep 10 13:50:16.345330 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:16.345001    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7841-88c29c" event={"ID":"779f1ba9-28b6-4d99-bf5c-aa15efaf5db5","Type":"ContainerStarted","Data":"b6a97d5f38e86cb9372281e235f798ad9c19b59590fc1a85c2fcc26da311381d"}
Sep 10 13:50:16.345330 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:16.345035    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7841-31c79b" podUID="35c78244-b747-4b89-a4fd-e404b49c8da9" containerName="test-curl-7841-31c79b" containerID="cri-o://6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1" gracePeriod=2
Sep 10 13:50:16.364604 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:16.364562    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7841-88c29c" podStartSLOduration=1.364548803 podStartE2EDuration="1.364548803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:50:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:50:16.364407675 +0000 UTC m=+1175.270965704" watchObservedRunningTime="2026-09-10 13:50:16.364548803 +0000 UTC m=+1175.271106854"
Sep 10 13:50:16.494469 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:16.494445    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-ca3a43"
Sep 10 13:50:16.562078 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:16.562052    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxnlc\" (UniqueName: \"kubernetes.io/projected/be228301-46a1-418b-8fec-ed796897fd9f-kube-api-access-bxnlc\") pod \"be228301-46a1-418b-8fec-ed796897fd9f\" (UID: \"be228301-46a1-418b-8fec-ed796897fd9f\") "
Sep 10 13:50:16.563982 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:16.563958    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be228301-46a1-418b-8fec-ed796897fd9f-kube-api-access-bxnlc" (OuterVolumeSpecName: "kube-api-access-bxnlc") pod "be228301-46a1-418b-8fec-ed796897fd9f" (UID: "be228301-46a1-418b-8fec-ed796897fd9f"). InnerVolumeSpecName "kube-api-access-bxnlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:50:16.662701 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:16.662677    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxnlc\" (UniqueName: \"kubernetes.io/projected/be228301-46a1-418b-8fec-ed796897fd9f-kube-api-access-bxnlc\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:50:17.351094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:17.351006    2549 generic.go:356] "Generic (PLEG): container finished" podID="be228301-46a1-418b-8fec-ed796897fd9f" containerID="6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded" exitCode=137
Sep 10 13:50:17.351094 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:17.351046    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-ca3a43"
Sep 10 13:50:17.351572 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:17.351116    2549 scope.go:117] "RemoveContainer" containerID="6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded"
Sep 10 13:50:17.363064 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:17.360337    2549 scope.go:117] "RemoveContainer" containerID="6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded"
Sep 10 13:50:17.363064 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:17.360611    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded\": container with ID starting with 6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded not found: ID does not exist" containerID="6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded"
Sep 10 13:50:17.363064 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:17.360646    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded"} err="failed to get container status \"6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded\": rpc error: code = NotFound desc = could not find container \"6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded\": container with ID starting with 6b151a41c2e3fd8facd90445bac55ff0423a9ddde815fe8547008f3fac253ded not found: ID does not exist"
Sep 10 13:50:17.365425 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:17.365383    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7841-ca3a43\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="be228301-46a1-418b-8fec-ed796897fd9f" pod="opendatahub/test-curl-7841-ca3a43"
Sep 10 13:50:17.386636 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:17.386601    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7841-88c29c"]
Sep 10 13:50:17.395681 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:17.395648    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7841-88c29c"]
Sep 10 13:50:17.674055 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:17.674015    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be228301-46a1-418b-8fec-ed796897fd9f" path="/var/lib/kubelet/pods/be228301-46a1-418b-8fec-ed796897fd9f/volumes"
Sep 10 13:50:18.356488 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:18.356458    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7841-88c29c" podUID="779f1ba9-28b6-4d99-bf5c-aa15efaf5db5" containerName="test-curl-7841-88c29c" containerID="cri-o://2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec" gracePeriod=2
Sep 10 13:50:18.540472 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:18.540446    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-31c79b"
Sep 10 13:50:18.579408 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:18.579382    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6555t\" (UniqueName: \"kubernetes.io/projected/35c78244-b747-4b89-a4fd-e404b49c8da9-kube-api-access-6555t\") pod \"35c78244-b747-4b89-a4fd-e404b49c8da9\" (UID: \"35c78244-b747-4b89-a4fd-e404b49c8da9\") "
Sep 10 13:50:18.581328 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:18.581303    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c78244-b747-4b89-a4fd-e404b49c8da9-kube-api-access-6555t" (OuterVolumeSpecName: "kube-api-access-6555t") pod "35c78244-b747-4b89-a4fd-e404b49c8da9" (UID: "35c78244-b747-4b89-a4fd-e404b49c8da9"). InnerVolumeSpecName "kube-api-access-6555t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:50:18.680859 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:18.680835    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6555t\" (UniqueName: \"kubernetes.io/projected/35c78244-b747-4b89-a4fd-e404b49c8da9-kube-api-access-6555t\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:50:19.361861 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:19.361828    2549 generic.go:356] "Generic (PLEG): container finished" podID="35c78244-b747-4b89-a4fd-e404b49c8da9" containerID="6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1" exitCode=137
Sep 10 13:50:19.362275 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:19.361896    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-31c79b"
Sep 10 13:50:19.362275 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:19.361936    2549 scope.go:117] "RemoveContainer" containerID="6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1"
Sep 10 13:50:19.370530 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:19.370514    2549 scope.go:117] "RemoveContainer" containerID="6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1"
Sep 10 13:50:19.370783 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:19.370763    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1\": container with ID starting with 6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1 not found: ID does not exist" containerID="6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1"
Sep 10 13:50:19.370826 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:19.370792    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1"} err="failed to get container status \"6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1\": rpc error: code = NotFound desc = could not find container \"6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1\": container with ID starting with 6936fb6d5ad0f63a46a02b0f18eb292f2064cf710942507fbecb773907c2aaf1 not found: ID does not exist"
Sep 10 13:50:19.372238 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:19.372214    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7841-31c79b\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="35c78244-b747-4b89-a4fd-e404b49c8da9" pod="opendatahub/test-curl-7841-31c79b"
Sep 10 13:50:19.673432 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:19.673402    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35c78244-b747-4b89-a4fd-e404b49c8da9" path="/var/lib/kubelet/pods/35c78244-b747-4b89-a4fd-e404b49c8da9/volumes"
Sep 10 13:50:20.538111 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:20.538087    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-88c29c"
Sep 10 13:50:20.597209 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:20.597187    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv557\" (UniqueName: \"kubernetes.io/projected/779f1ba9-28b6-4d99-bf5c-aa15efaf5db5-kube-api-access-pv557\") pod \"779f1ba9-28b6-4d99-bf5c-aa15efaf5db5\" (UID: \"779f1ba9-28b6-4d99-bf5c-aa15efaf5db5\") "
Sep 10 13:50:20.599089 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:20.599062    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/779f1ba9-28b6-4d99-bf5c-aa15efaf5db5-kube-api-access-pv557" (OuterVolumeSpecName: "kube-api-access-pv557") pod "779f1ba9-28b6-4d99-bf5c-aa15efaf5db5" (UID: "779f1ba9-28b6-4d99-bf5c-aa15efaf5db5"). InnerVolumeSpecName "kube-api-access-pv557". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:50:20.698282 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:20.698256    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv557\" (UniqueName: \"kubernetes.io/projected/779f1ba9-28b6-4d99-bf5c-aa15efaf5db5-kube-api-access-pv557\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:50:21.378149 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:21.378113    2549 generic.go:356] "Generic (PLEG): container finished" podID="779f1ba9-28b6-4d99-bf5c-aa15efaf5db5" containerID="2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec" exitCode=137
Sep 10 13:50:21.378324 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:21.378156    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7841-88c29c"
Sep 10 13:50:21.378324 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:21.378211    2549 scope.go:117] "RemoveContainer" containerID="2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec"
Sep 10 13:50:21.386959 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:21.386945    2549 scope.go:117] "RemoveContainer" containerID="2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec"
Sep 10 13:50:21.387204 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:21.387189    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec\": container with ID starting with 2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec not found: ID does not exist" containerID="2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec"
Sep 10 13:50:21.387266 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:21.387211    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec"} err="failed to get container status \"2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec\": rpc error: code = NotFound desc = could not find container \"2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec\": container with ID starting with 2dd51626a9c4331760facdabe4737f77ea20776b091fc694427d48bb6c3536ec not found: ID does not exist"
Sep 10 13:50:21.388390 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:21.388368    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7841-88c29c\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="779f1ba9-28b6-4d99-bf5c-aa15efaf5db5" pod="opendatahub/test-curl-7841-88c29c"
Sep 10 13:50:21.672601 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:50:21.672567    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7841-88c29c\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="779f1ba9-28b6-4d99-bf5c-aa15efaf5db5" pod="opendatahub/test-curl-7841-88c29c"
Sep 10 13:50:21.674520 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:21.674496    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="779f1ba9-28b6-4d99-bf5c-aa15efaf5db5" path="/var/lib/kubelet/pods/779f1ba9-28b6-4d99-bf5c-aa15efaf5db5/volumes"
Sep 10 13:50:25.340193 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:25.340161    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 13:50:28.888111 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:28.888078    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"]
Sep 10 13:50:28.888490 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:28.888390    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="779f1ba9-28b6-4d99-bf5c-aa15efaf5db5" containerName="test-curl-7841-88c29c"
Sep 10 13:50:28.888490 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:28.888400    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="779f1ba9-28b6-4d99-bf5c-aa15efaf5db5" containerName="test-curl-7841-88c29c"
Sep 10 13:50:28.888490 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:28.888454    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="779f1ba9-28b6-4d99-bf5c-aa15efaf5db5" containerName="test-curl-7841-88c29c"
Sep 10 13:50:28.892656 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:28.892639    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:50:28.895829 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:28.895811    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-a6883ccf\""
Sep 10 13:50:28.895958 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:28.895858    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-a6883ccf-dockercfg-jqmmn\""
Sep 10 13:50:28.907759 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:28.907730    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"]
Sep 10 13:50:28.961623 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:28.961592    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ec48c8-5b06-49f9-8280-3ae80a902be7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb\" (UID: \"b5ec48c8-5b06-49f9-8280-3ae80a902be7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:50:28.961767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:28.961641    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7rks\" (UniqueName: \"kubernetes.io/projected/b5ec48c8-5b06-49f9-8280-3ae80a902be7-kube-api-access-z7rks\") pod \"payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb\" (UID: \"b5ec48c8-5b06-49f9-8280-3ae80a902be7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:50:29.017496 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.017469    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"]
Sep 10 13:50:29.020635 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.020620    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:50:29.042716 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.042691    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"]
Sep 10 13:50:29.062219 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.062190    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ec48c8-5b06-49f9-8280-3ae80a902be7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb\" (UID: \"b5ec48c8-5b06-49f9-8280-3ae80a902be7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:50:29.062341 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.062256    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7rks\" (UniqueName: \"kubernetes.io/projected/b5ec48c8-5b06-49f9-8280-3ae80a902be7-kube-api-access-z7rks\") pod \"payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb\" (UID: \"b5ec48c8-5b06-49f9-8280-3ae80a902be7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:50:29.062849 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.062832    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ec48c8-5b06-49f9-8280-3ae80a902be7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb\" (UID: \"b5ec48c8-5b06-49f9-8280-3ae80a902be7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:50:29.079807 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.079781    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7rks\" (UniqueName: \"kubernetes.io/projected/b5ec48c8-5b06-49f9-8280-3ae80a902be7-kube-api-access-z7rks\") pod \"payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb\" (UID: \"b5ec48c8-5b06-49f9-8280-3ae80a902be7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:50:29.162732 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.162704    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b6027a5-9834-4bd8-a48c-2478cf8ca998-plugins-config-volume\") pod \"payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn\" (UID: \"2b6027a5-9834-4bd8-a48c-2478cf8ca998\") " pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:50:29.162887 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.162744    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp9zd\" (UniqueName: \"kubernetes.io/projected/2b6027a5-9834-4bd8-a48c-2478cf8ca998-kube-api-access-dp9zd\") pod \"payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn\" (UID: \"2b6027a5-9834-4bd8-a48c-2478cf8ca998\") " pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:50:29.201585 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.201556    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:50:29.263551 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.263502    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b6027a5-9834-4bd8-a48c-2478cf8ca998-plugins-config-volume\") pod \"payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn\" (UID: \"2b6027a5-9834-4bd8-a48c-2478cf8ca998\") " pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:50:29.263699 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.263568    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp9zd\" (UniqueName: \"kubernetes.io/projected/2b6027a5-9834-4bd8-a48c-2478cf8ca998-kube-api-access-dp9zd\") pod \"payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn\" (UID: \"2b6027a5-9834-4bd8-a48c-2478cf8ca998\") " pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:50:29.264723 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.264691    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b6027a5-9834-4bd8-a48c-2478cf8ca998-plugins-config-volume\") pod \"payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn\" (UID: \"2b6027a5-9834-4bd8-a48c-2478cf8ca998\") " pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:50:29.274741 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.274709    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp9zd\" (UniqueName: \"kubernetes.io/projected/2b6027a5-9834-4bd8-a48c-2478cf8ca998-kube-api-access-dp9zd\") pod \"payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn\" (UID: \"2b6027a5-9834-4bd8-a48c-2478cf8ca998\") " pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:50:29.330324 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.330296    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:50:29.480141 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.480059    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"]
Sep 10 13:50:29.483039 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:50:29.483013    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ec48c8_5b06_49f9_8280_3ae80a902be7.slice/crio-174ea720887dc805152b460e9620f9516d7fee2463344c1e3da48d0422127cb4 WatchSource:0}: Error finding container 174ea720887dc805152b460e9620f9516d7fee2463344c1e3da48d0422127cb4: Status 404 returned error can't find the container with id 174ea720887dc805152b460e9620f9516d7fee2463344c1e3da48d0422127cb4
Sep 10 13:50:29.608101 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:29.608078    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"]
Sep 10 13:50:29.610447 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:50:29.610423    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6027a5_9834_4bd8_a48c_2478cf8ca998.slice/crio-3df171c1198460aff4d9c1714b288d9b3930fd2d093db9195fde7b3658e21bf3 WatchSource:0}: Error finding container 3df171c1198460aff4d9c1714b288d9b3930fd2d093db9195fde7b3658e21bf3: Status 404 returned error can't find the container with id 3df171c1198460aff4d9c1714b288d9b3930fd2d093db9195fde7b3658e21bf3
Sep 10 13:50:30.428213 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:30.428177    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn" event={"ID":"2b6027a5-9834-4bd8-a48c-2478cf8ca998","Type":"ContainerStarted","Data":"8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b"}
Sep 10 13:50:30.428670 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:30.428229    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn" event={"ID":"2b6027a5-9834-4bd8-a48c-2478cf8ca998","Type":"ContainerStarted","Data":"3df171c1198460aff4d9c1714b288d9b3930fd2d093db9195fde7b3658e21bf3"}
Sep 10 13:50:30.428670 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:30.428285    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:50:30.429819 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:30.429792    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb" event={"ID":"b5ec48c8-5b06-49f9-8280-3ae80a902be7","Type":"ContainerStarted","Data":"801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8"}
Sep 10 13:50:30.429948 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:30.429836    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:50:30.429948 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:30.429851    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb" event={"ID":"b5ec48c8-5b06-49f9-8280-3ae80a902be7","Type":"ContainerStarted","Data":"174ea720887dc805152b460e9620f9516d7fee2463344c1e3da48d0422127cb4"}
Sep 10 13:50:30.488850 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:30.488808    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn" podStartSLOduration=2.488794493 podStartE2EDuration="2.488794493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:50:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:50:30.457986667 +0000 UTC m=+1189.364544698" watchObservedRunningTime="2026-09-10 13:50:30.488794493 +0000 UTC m=+1189.395352524"
Sep 10 13:50:30.489468 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:30.489443    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb" podStartSLOduration=2.4894363139999998 podStartE2EDuration="2.489436314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:50:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:50:30.487853125 +0000 UTC m=+1189.394411167" watchObservedRunningTime="2026-09-10 13:50:30.489436314 +0000 UTC m=+1189.395994344"
Sep 10 13:50:41.435542 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:41.435506    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:50:41.435988 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:41.435678    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:50:41.561034 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:41.561003    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:50:41.562894 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:50:41.562855    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:51:41.658033 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:41.657967    2549 scope.go:117] "RemoveContainer" containerID="647c3ec40f65b9d3dcfd04a9bef9cc9015c3607d1dbfac3de468f47d04af91ed"
Sep 10 13:51:42.696955 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.696925    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"]
Sep 10 13:51:42.701439 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.701422    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.704028 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.704005    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a6883ccf\"/\"kube-root-ca.crt\""
Sep 10 13:51:42.705138 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.705118    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a6883ccf\"/\"default-dockercfg-cjn78\""
Sep 10 13:51:42.705233 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.705140    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a6883ccf\"/\"openshift-service-ca.crt\""
Sep 10 13:51:42.705233 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.705183    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a6883ccf\"/\"e2e-lifecycle-model-a6883ccf-kserve-self-signed-certs\""
Sep 10 13:51:42.711389 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.711367    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"]
Sep 10 13:51:42.795853 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.795822    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-home\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.796033 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.795927    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71249a71-cf61-4d0d-bc56-af89c6619dd2-tls-certs\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.796033 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.795952    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89chf\" (UniqueName: \"kubernetes.io/projected/71249a71-cf61-4d0d-bc56-af89c6619dd2-kube-api-access-89chf\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.796033 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.795993    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-tmp-dir\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.796033 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.796018    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-model-cache\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.796214 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.796041    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-dshm\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.897228 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.897193    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71249a71-cf61-4d0d-bc56-af89c6619dd2-tls-certs\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.897395 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.897235    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89chf\" (UniqueName: \"kubernetes.io/projected/71249a71-cf61-4d0d-bc56-af89c6619dd2-kube-api-access-89chf\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.897395 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.897257    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-tmp-dir\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.897395 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.897282    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-model-cache\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.897395 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.897302    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-dshm\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.897395 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.897322    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-home\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.897749 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.897726    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-model-cache\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.897843 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.897767    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-tmp-dir\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.897843 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.897812    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-home\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.899576 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.899554    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-dshm\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.899738 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.899722    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71249a71-cf61-4d0d-bc56-af89c6619dd2-tls-certs\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:42.907904 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:42.907881    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89chf\" (UniqueName: \"kubernetes.io/projected/71249a71-cf61-4d0d-bc56-af89c6619dd2-kube-api-access-89chf\") pod \"e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") " pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:43.010938 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:43.010826    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:43.284591 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:43.284565    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"]
Sep 10 13:51:43.286431 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:51:43.286401    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71249a71_cf61_4d0d_bc56_af89c6619dd2.slice/crio-37828e9f262a7278875ad03c937bbe031af0e7a4299a8a76770a4926faf32ab7 WatchSource:0}: Error finding container 37828e9f262a7278875ad03c937bbe031af0e7a4299a8a76770a4926faf32ab7: Status 404 returned error can't find the container with id 37828e9f262a7278875ad03c937bbe031af0e7a4299a8a76770a4926faf32ab7
Sep 10 13:51:43.763043 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:43.762998    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8" event={"ID":"71249a71-cf61-4d0d-bc56-af89c6619dd2","Type":"ContainerStarted","Data":"df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923"}
Sep 10 13:51:43.763411 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:43.763050    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8" event={"ID":"71249a71-cf61-4d0d-bc56-af89c6619dd2","Type":"ContainerStarted","Data":"37828e9f262a7278875ad03c937bbe031af0e7a4299a8a76770a4926faf32ab7"}
Sep 10 13:51:43.782879 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:43.782816    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8" podStartSLOduration=1.7828015270000002 podStartE2EDuration="1.782801527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:51:43.781858728 +0000 UTC m=+1262.688416757" watchObservedRunningTime="2026-09-10 13:51:43.782801527 +0000 UTC m=+1262.689359592"
Sep 10 13:51:53.011964 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:53.011923    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:53.012569 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:53.012224    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:53.024724 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:53.024694    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:51:53.814828 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:51:53.814795    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:52:27.697999 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:27.697964    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"]
Sep 10 13:52:27.701595 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:27.701568    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"
Sep 10 13:52:27.704640 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:27.704618    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:52:27.704758 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:27.704721    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 13:52:27.711200 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:27.711166    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"]
Sep 10 13:52:27.895700 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:27.895663    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f73066e1-c674-427c-b77c-b75ffd7f95c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a6883ccf-l974p\" (UID: \"f73066e1-c674-427c-b77c-b75ffd7f95c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"
Sep 10 13:52:27.996654 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:27.996557    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f73066e1-c674-427c-b77c-b75ffd7f95c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a6883ccf-l974p\" (UID: \"f73066e1-c674-427c-b77c-b75ffd7f95c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"
Sep 10 13:52:27.997291 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:27.997262    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f73066e1-c674-427c-b77c-b75ffd7f95c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a6883ccf-l974p\" (UID: \"f73066e1-c674-427c-b77c-b75ffd7f95c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"
Sep 10 13:52:28.013575 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:28.013535    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"
Sep 10 13:52:28.290762 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:28.290731    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"]
Sep 10 13:52:28.293276 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:52:28.293245    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73066e1_c674_427c_b77c_b75ffd7f95c6.slice/crio-56c813649f3765b85723c18353c44a7ba3a681160be722cf5b3ca08cae26b8de WatchSource:0}: Error finding container 56c813649f3765b85723c18353c44a7ba3a681160be722cf5b3ca08cae26b8de: Status 404 returned error can't find the container with id 56c813649f3765b85723c18353c44a7ba3a681160be722cf5b3ca08cae26b8de
Sep 10 13:52:28.963086 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:28.963052    2549 generic.go:356] "Generic (PLEG): container finished" podID="f73066e1-c674-427c-b77c-b75ffd7f95c6" containerID="73689f3e15874f86e49425de52429b0d204f167e3c02d61dea966b9a05c894d3" exitCode=0
Sep 10 13:52:28.963473 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:28.963135    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p" event={"ID":"f73066e1-c674-427c-b77c-b75ffd7f95c6","Type":"ContainerDied","Data":"73689f3e15874f86e49425de52429b0d204f167e3c02d61dea966b9a05c894d3"}
Sep 10 13:52:28.963473 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:28.963187    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p" event={"ID":"f73066e1-c674-427c-b77c-b75ffd7f95c6","Type":"ContainerStarted","Data":"56c813649f3765b85723c18353c44a7ba3a681160be722cf5b3ca08cae26b8de"}
Sep 10 13:52:30.097728 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:30.097707    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"
Sep 10 13:52:30.115478 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:30.115445    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f73066e1-c674-427c-b77c-b75ffd7f95c6-maas-api-service-ca\") pod \"f73066e1-c674-427c-b77c-b75ffd7f95c6\" (UID: \"f73066e1-c674-427c-b77c-b75ffd7f95c6\") "
Sep 10 13:52:30.115776 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:30.115753    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f73066e1-c674-427c-b77c-b75ffd7f95c6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f73066e1-c674-427c-b77c-b75ffd7f95c6" (UID: "f73066e1-c674-427c-b77c-b75ffd7f95c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:52:30.216079 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:30.216044    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f73066e1-c674-427c-b77c-b75ffd7f95c6-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:52:30.979914 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:30.979883    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"
Sep 10 13:52:30.980084 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:30.979922    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p" event={"ID":"f73066e1-c674-427c-b77c-b75ffd7f95c6","Type":"ContainerDied","Data":"56c813649f3765b85723c18353c44a7ba3a681160be722cf5b3ca08cae26b8de"}
Sep 10 13:52:30.980084 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:30.979988    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56c813649f3765b85723c18353c44a7ba3a681160be722cf5b3ca08cae26b8de"
Sep 10 13:52:43.453979 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.453945    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"]
Sep 10 13:52:43.454466 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.454162    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn" podUID="2b6027a5-9834-4bd8-a48c-2478cf8ca998" containerName="payload-processing" containerID="cri-o://8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b" gracePeriod=30
Sep 10 13:52:43.460994 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.460965    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"]
Sep 10 13:52:43.461199 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.461171    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb" podUID="b5ec48c8-5b06-49f9-8280-3ae80a902be7" containerName="payload-pre-processing" containerID="cri-o://801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8" gracePeriod=30
Sep 10 13:52:43.683917 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.683885    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"]
Sep 10 13:52:43.693628 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.693597    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6883ccf-l974p"]
Sep 10 13:52:43.733629 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.733606    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:52:43.736135 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.736116    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:52:43.832205 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.832168    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp9zd\" (UniqueName: \"kubernetes.io/projected/2b6027a5-9834-4bd8-a48c-2478cf8ca998-kube-api-access-dp9zd\") pod \"2b6027a5-9834-4bd8-a48c-2478cf8ca998\" (UID: \"2b6027a5-9834-4bd8-a48c-2478cf8ca998\") "
Sep 10 13:52:43.832205 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.832216    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b6027a5-9834-4bd8-a48c-2478cf8ca998-plugins-config-volume\") pod \"2b6027a5-9834-4bd8-a48c-2478cf8ca998\" (UID: \"2b6027a5-9834-4bd8-a48c-2478cf8ca998\") "
Sep 10 13:52:43.832429 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.832237    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7rks\" (UniqueName: \"kubernetes.io/projected/b5ec48c8-5b06-49f9-8280-3ae80a902be7-kube-api-access-z7rks\") pod \"b5ec48c8-5b06-49f9-8280-3ae80a902be7\" (UID: \"b5ec48c8-5b06-49f9-8280-3ae80a902be7\") "
Sep 10 13:52:43.832429 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.832326    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ec48c8-5b06-49f9-8280-3ae80a902be7-plugins-config-volume\") pod \"b5ec48c8-5b06-49f9-8280-3ae80a902be7\" (UID: \"b5ec48c8-5b06-49f9-8280-3ae80a902be7\") "
Sep 10 13:52:43.832674 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.832651    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b6027a5-9834-4bd8-a48c-2478cf8ca998-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b6027a5-9834-4bd8-a48c-2478cf8ca998" (UID: "2b6027a5-9834-4bd8-a48c-2478cf8ca998"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:52:43.832739 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.832706    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5ec48c8-5b06-49f9-8280-3ae80a902be7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5ec48c8-5b06-49f9-8280-3ae80a902be7" (UID: "b5ec48c8-5b06-49f9-8280-3ae80a902be7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:52:43.834421 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.834397    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b6027a5-9834-4bd8-a48c-2478cf8ca998-kube-api-access-dp9zd" (OuterVolumeSpecName: "kube-api-access-dp9zd") pod "2b6027a5-9834-4bd8-a48c-2478cf8ca998" (UID: "2b6027a5-9834-4bd8-a48c-2478cf8ca998"). InnerVolumeSpecName "kube-api-access-dp9zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:52:43.834488 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.834431    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ec48c8-5b06-49f9-8280-3ae80a902be7-kube-api-access-z7rks" (OuterVolumeSpecName: "kube-api-access-z7rks") pod "b5ec48c8-5b06-49f9-8280-3ae80a902be7" (UID: "b5ec48c8-5b06-49f9-8280-3ae80a902be7"). InnerVolumeSpecName "kube-api-access-z7rks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:52:43.933162 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.933126    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp9zd\" (UniqueName: \"kubernetes.io/projected/2b6027a5-9834-4bd8-a48c-2478cf8ca998-kube-api-access-dp9zd\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:52:43.933162 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.933157    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b6027a5-9834-4bd8-a48c-2478cf8ca998-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:52:43.933162 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.933168    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7rks\" (UniqueName: \"kubernetes.io/projected/b5ec48c8-5b06-49f9-8280-3ae80a902be7-kube-api-access-z7rks\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:52:43.933404 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:43.933177    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ec48c8-5b06-49f9-8280-3ae80a902be7-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:52:44.044980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.044892    2549 generic.go:356] "Generic (PLEG): container finished" podID="2b6027a5-9834-4bd8-a48c-2478cf8ca998" containerID="8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b" exitCode=0
Sep 10 13:52:44.044980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.044930    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"
Sep 10 13:52:44.044980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.044956    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn" event={"ID":"2b6027a5-9834-4bd8-a48c-2478cf8ca998","Type":"ContainerDied","Data":"8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b"}
Sep 10 13:52:44.045233 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.045013    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn" event={"ID":"2b6027a5-9834-4bd8-a48c-2478cf8ca998","Type":"ContainerDied","Data":"3df171c1198460aff4d9c1714b288d9b3930fd2d093db9195fde7b3658e21bf3"}
Sep 10 13:52:44.045233 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.045046    2549 scope.go:117] "RemoveContainer" containerID="8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b"
Sep 10 13:52:44.046386 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.046362    2549 generic.go:356] "Generic (PLEG): container finished" podID="b5ec48c8-5b06-49f9-8280-3ae80a902be7" containerID="801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8" exitCode=0
Sep 10 13:52:44.046500 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.046404    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"
Sep 10 13:52:44.046500 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.046430    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb" event={"ID":"b5ec48c8-5b06-49f9-8280-3ae80a902be7","Type":"ContainerDied","Data":"801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8"}
Sep 10 13:52:44.046500 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.046483    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb" event={"ID":"b5ec48c8-5b06-49f9-8280-3ae80a902be7","Type":"ContainerDied","Data":"174ea720887dc805152b460e9620f9516d7fee2463344c1e3da48d0422127cb4"}
Sep 10 13:52:44.053893 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.053846    2549 scope.go:117] "RemoveContainer" containerID="8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b"
Sep 10 13:52:44.054244 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:52:44.054223    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b\": container with ID starting with 8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b not found: ID does not exist" containerID="8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b"
Sep 10 13:52:44.054349 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.054255    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b"} err="failed to get container status \"8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b\": rpc error: code = NotFound desc = could not find container \"8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b\": container with ID starting with 8e0a4bd1488c74dace9836452da156dbba06efa9d59599cce452b485688fab8b not found: ID does not exist"
Sep 10 13:52:44.054349 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.054279    2549 scope.go:117] "RemoveContainer" containerID="801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8"
Sep 10 13:52:44.062262 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.062243    2549 scope.go:117] "RemoveContainer" containerID="801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8"
Sep 10 13:52:44.062523 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:52:44.062505    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8\": container with ID starting with 801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8 not found: ID does not exist" containerID="801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8"
Sep 10 13:52:44.062571 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.062531    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8"} err="failed to get container status \"801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8\": rpc error: code = NotFound desc = could not find container \"801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8\": container with ID starting with 801c318d16a0e27846f257eb8cfc4fa3f67ce601aed78b1d0ee1f0fc6c6115f8 not found: ID does not exist"
Sep 10 13:52:44.073087 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.073065    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"]
Sep 10 13:52:44.076038 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.076018    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a6883ccf-779cb5cb94-ltxxb"]
Sep 10 13:52:44.092925 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.092898    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"]
Sep 10 13:52:44.094224 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:44.094206    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a6883ccf-847569cc57-hcpnn"]
Sep 10 13:52:45.673881 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:45.673845    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b6027a5-9834-4bd8-a48c-2478cf8ca998" path="/var/lib/kubelet/pods/2b6027a5-9834-4bd8-a48c-2478cf8ca998/volumes"
Sep 10 13:52:45.674240 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:45.674215    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5ec48c8-5b06-49f9-8280-3ae80a902be7" path="/var/lib/kubelet/pods/b5ec48c8-5b06-49f9-8280-3ae80a902be7/volumes"
Sep 10 13:52:45.674494 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:45.674481    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f73066e1-c674-427c-b77c-b75ffd7f95c6" path="/var/lib/kubelet/pods/f73066e1-c674-427c-b77c-b75ffd7f95c6/volumes"
Sep 10 13:52:52.085306 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:52:52.085272    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"]
Sep 10 13:53:07.104512 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.104406    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8" podUID="71249a71-cf61-4d0d-bc56-af89c6619dd2" containerName="main" containerID="cri-o://df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923" gracePeriod=45
Sep 10 13:53:07.346277 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.346253    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:53:07.442811 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.442774    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-model-cache\") pod \"71249a71-cf61-4d0d-bc56-af89c6619dd2\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") "
Sep 10 13:53:07.443019 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.442822    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-dshm\") pod \"71249a71-cf61-4d0d-bc56-af89c6619dd2\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") "
Sep 10 13:53:07.443019 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.442883    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71249a71-cf61-4d0d-bc56-af89c6619dd2-tls-certs\") pod \"71249a71-cf61-4d0d-bc56-af89c6619dd2\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") "
Sep 10 13:53:07.443019 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.442910    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-home\") pod \"71249a71-cf61-4d0d-bc56-af89c6619dd2\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") "
Sep 10 13:53:07.443019 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.442940    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89chf\" (UniqueName: \"kubernetes.io/projected/71249a71-cf61-4d0d-bc56-af89c6619dd2-kube-api-access-89chf\") pod \"71249a71-cf61-4d0d-bc56-af89c6619dd2\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") "
Sep 10 13:53:07.443019 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.442994    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-tmp-dir\") pod \"71249a71-cf61-4d0d-bc56-af89c6619dd2\" (UID: \"71249a71-cf61-4d0d-bc56-af89c6619dd2\") "
Sep 10 13:53:07.443225 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.443048    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-model-cache" (OuterVolumeSpecName: "model-cache") pod "71249a71-cf61-4d0d-bc56-af89c6619dd2" (UID: "71249a71-cf61-4d0d-bc56-af89c6619dd2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:53:07.443603 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.443484    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-home" (OuterVolumeSpecName: "home") pod "71249a71-cf61-4d0d-bc56-af89c6619dd2" (UID: "71249a71-cf61-4d0d-bc56-af89c6619dd2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:53:07.443603 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.443597    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:53:07.444983 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.443741    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71249a71-cf61-4d0d-bc56-af89c6619dd2" (UID: "71249a71-cf61-4d0d-bc56-af89c6619dd2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:53:07.446090 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.446031    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-dshm" (OuterVolumeSpecName: "dshm") pod "71249a71-cf61-4d0d-bc56-af89c6619dd2" (UID: "71249a71-cf61-4d0d-bc56-af89c6619dd2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:53:07.446195 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.446114    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71249a71-cf61-4d0d-bc56-af89c6619dd2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71249a71-cf61-4d0d-bc56-af89c6619dd2" (UID: "71249a71-cf61-4d0d-bc56-af89c6619dd2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:53:07.446195 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.446134    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71249a71-cf61-4d0d-bc56-af89c6619dd2-kube-api-access-89chf" (OuterVolumeSpecName: "kube-api-access-89chf") pod "71249a71-cf61-4d0d-bc56-af89c6619dd2" (UID: "71249a71-cf61-4d0d-bc56-af89c6619dd2"). InnerVolumeSpecName "kube-api-access-89chf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:53:07.544545 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.544498    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:53:07.544545 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.544539    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:53:07.544545 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.544548    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71249a71-cf61-4d0d-bc56-af89c6619dd2-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:53:07.544545 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.544556    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71249a71-cf61-4d0d-bc56-af89c6619dd2-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:53:07.544790 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:07.544564    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89chf\" (UniqueName: \"kubernetes.io/projected/71249a71-cf61-4d0d-bc56-af89c6619dd2-kube-api-access-89chf\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:53:08.155960 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:08.155922    2549 generic.go:356] "Generic (PLEG): container finished" podID="71249a71-cf61-4d0d-bc56-af89c6619dd2" containerID="df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923" exitCode=0
Sep 10 13:53:08.155960 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:08.155963    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8" event={"ID":"71249a71-cf61-4d0d-bc56-af89c6619dd2","Type":"ContainerDied","Data":"df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923"}
Sep 10 13:53:08.156477 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:08.155983    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"
Sep 10 13:53:08.156477 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:08.156003    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8" event={"ID":"71249a71-cf61-4d0d-bc56-af89c6619dd2","Type":"ContainerDied","Data":"37828e9f262a7278875ad03c937bbe031af0e7a4299a8a76770a4926faf32ab7"}
Sep 10 13:53:08.156477 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:08.156024    2549 scope.go:117] "RemoveContainer" containerID="df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923"
Sep 10 13:53:08.164590 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:08.164575    2549 scope.go:117] "RemoveContainer" containerID="df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923"
Sep 10 13:53:08.164847 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:53:08.164831    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923\": container with ID starting with df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923 not found: ID does not exist" containerID="df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923"
Sep 10 13:53:08.164927 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:08.164855    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923"} err="failed to get container status \"df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923\": rpc error: code = NotFound desc = could not find container \"df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923\": container with ID starting with df275c3b3cb25d96eadc0200d6f5f2a1c1867dfebeebff0cf41372cde47b5923 not found: ID does not exist"
Sep 10 13:53:08.174438 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:08.174413    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"]
Sep 10 13:53:08.179581 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:08.179558    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-a6883ccf/e2e-lifecycle-model-a6883ccf-kserve-586cb88b67-4crc8"]
Sep 10 13:53:09.674072 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:53:09.674038    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71249a71-cf61-4d0d-bc56-af89c6619dd2" path="/var/lib/kubelet/pods/71249a71-cf61-4d0d-bc56-af89c6619dd2/volumes"
Sep 10 13:54:02.693994 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.693961    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7844-038158"]
Sep 10 13:54:02.694384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694286    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73066e1-c674-427c-b77c-b75ffd7f95c6" containerName="revoke-keys"
Sep 10 13:54:02.694384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694297    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73066e1-c674-427c-b77c-b75ffd7f95c6" containerName="revoke-keys"
Sep 10 13:54:02.694384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694315    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ec48c8-5b06-49f9-8280-3ae80a902be7" containerName="payload-pre-processing"
Sep 10 13:54:02.694384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694320    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ec48c8-5b06-49f9-8280-3ae80a902be7" containerName="payload-pre-processing"
Sep 10 13:54:02.694384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694327    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b6027a5-9834-4bd8-a48c-2478cf8ca998" containerName="payload-processing"
Sep 10 13:54:02.694384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694332    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b6027a5-9834-4bd8-a48c-2478cf8ca998" containerName="payload-processing"
Sep 10 13:54:02.694384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694340    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71249a71-cf61-4d0d-bc56-af89c6619dd2" containerName="main"
Sep 10 13:54:02.694384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694346    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="71249a71-cf61-4d0d-bc56-af89c6619dd2" containerName="main"
Sep 10 13:54:02.694644 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694402    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="f73066e1-c674-427c-b77c-b75ffd7f95c6" containerName="revoke-keys"
Sep 10 13:54:02.694644 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694409    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5ec48c8-5b06-49f9-8280-3ae80a902be7" containerName="payload-pre-processing"
Sep 10 13:54:02.694644 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694418    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b6027a5-9834-4bd8-a48c-2478cf8ca998" containerName="payload-processing"
Sep 10 13:54:02.694644 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.694423    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="71249a71-cf61-4d0d-bc56-af89c6619dd2" containerName="main"
Sep 10 13:54:02.698465 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.698448    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-038158"
Sep 10 13:54:02.701187 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.701164    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-g8rj6\""
Sep 10 13:54:02.705150 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.705124    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7844-038158"]
Sep 10 13:54:02.803948 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.803915    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5b9g\" (UniqueName: \"kubernetes.io/projected/b8acebaa-c492-4a43-b23e-2a9eb3deec2b-kube-api-access-c5b9g\") pod \"test-curl-7844-038158\" (UID: \"b8acebaa-c492-4a43-b23e-2a9eb3deec2b\") " pod="opendatahub/test-curl-7844-038158"
Sep 10 13:54:02.905167 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.905134    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5b9g\" (UniqueName: \"kubernetes.io/projected/b8acebaa-c492-4a43-b23e-2a9eb3deec2b-kube-api-access-c5b9g\") pod \"test-curl-7844-038158\" (UID: \"b8acebaa-c492-4a43-b23e-2a9eb3deec2b\") " pod="opendatahub/test-curl-7844-038158"
Sep 10 13:54:02.914407 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:02.914372    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5b9g\" (UniqueName: \"kubernetes.io/projected/b8acebaa-c492-4a43-b23e-2a9eb3deec2b-kube-api-access-c5b9g\") pod \"test-curl-7844-038158\" (UID: \"b8acebaa-c492-4a43-b23e-2a9eb3deec2b\") " pod="opendatahub/test-curl-7844-038158"
Sep 10 13:54:03.008279 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:03.008196    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-038158"
Sep 10 13:54:03.287895 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:03.287787    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7844-038158"]
Sep 10 13:54:03.410476 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:03.410437    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7844-038158" event={"ID":"b8acebaa-c492-4a43-b23e-2a9eb3deec2b","Type":"ContainerStarted","Data":"e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8"}
Sep 10 13:54:03.410655 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:03.410485    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7844-038158" event={"ID":"b8acebaa-c492-4a43-b23e-2a9eb3deec2b","Type":"ContainerStarted","Data":"3a4d38dde1646254d7eb554c8031b00c04106a43d0b881ebbb8d1c6d14779828"}
Sep 10 13:54:03.436392 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:03.436341    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7844-038158" podStartSLOduration=1.4362994150000001 podStartE2EDuration="1.436299415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:54:03.434711612 +0000 UTC m=+1402.341269646" watchObservedRunningTime="2026-09-10 13:54:03.436299415 +0000 UTC m=+1402.342857446"
Sep 10 13:54:04.045818 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:04.045766    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7844-038158"]
Sep 10 13:54:04.053060 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:04.053031    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7844-038158"]
Sep 10 13:54:04.415193 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:54:04.415164    2549 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7844-038158" container="test-curl-7844-038158"
Sep 10 13:54:05.418835 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:05.418792    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7844-038158" podUID="b8acebaa-c492-4a43-b23e-2a9eb3deec2b" containerName="test-curl-7844-038158" containerID="cri-o://e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8" gracePeriod=2
Sep 10 13:54:05.421373 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:54:05.421328    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7844-038158\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="b8acebaa-c492-4a43-b23e-2a9eb3deec2b" pod="opendatahub/test-curl-7844-038158"
Sep 10 13:54:07.591803 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:07.591781    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-038158"
Sep 10 13:54:07.638207 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:07.638179    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5b9g\" (UniqueName: \"kubernetes.io/projected/b8acebaa-c492-4a43-b23e-2a9eb3deec2b-kube-api-access-c5b9g\") pod \"b8acebaa-c492-4a43-b23e-2a9eb3deec2b\" (UID: \"b8acebaa-c492-4a43-b23e-2a9eb3deec2b\") "
Sep 10 13:54:07.640191 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:07.640162    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8acebaa-c492-4a43-b23e-2a9eb3deec2b-kube-api-access-c5b9g" (OuterVolumeSpecName: "kube-api-access-c5b9g") pod "b8acebaa-c492-4a43-b23e-2a9eb3deec2b" (UID: "b8acebaa-c492-4a43-b23e-2a9eb3deec2b"). InnerVolumeSpecName "kube-api-access-c5b9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:54:07.673910 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:07.673812    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8acebaa-c492-4a43-b23e-2a9eb3deec2b" path="/var/lib/kubelet/pods/b8acebaa-c492-4a43-b23e-2a9eb3deec2b/volumes"
Sep 10 13:54:07.739663 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:07.739621    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5b9g\" (UniqueName: \"kubernetes.io/projected/b8acebaa-c492-4a43-b23e-2a9eb3deec2b-kube-api-access-c5b9g\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:54:08.433759 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:08.433720    2549 generic.go:356] "Generic (PLEG): container finished" podID="b8acebaa-c492-4a43-b23e-2a9eb3deec2b" containerID="e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8" exitCode=137
Sep 10 13:54:08.433759 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:08.433763    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-038158"
Sep 10 13:54:08.434023 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:08.433833    2549 scope.go:117] "RemoveContainer" containerID="e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8"
Sep 10 13:54:08.442542 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:08.442518    2549 scope.go:117] "RemoveContainer" containerID="e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8"
Sep 10 13:54:08.442952 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:54:08.442930    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8\": container with ID starting with e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8 not found: ID does not exist" containerID="e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8"
Sep 10 13:54:08.443027 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:08.442963    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8"} err="failed to get container status \"e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8\": rpc error: code = NotFound desc = could not find container \"e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8\": container with ID starting with e570169cebc6ee4da9b829729cc5a71bfbb45d406e76b3dc76653a25c41dfde8 not found: ID does not exist"
Sep 10 13:54:30.694836 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.694799    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"]
Sep 10 13:54:30.695266 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.695153    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8acebaa-c492-4a43-b23e-2a9eb3deec2b" containerName="test-curl-7844-038158"
Sep 10 13:54:30.695266 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.695164    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8acebaa-c492-4a43-b23e-2a9eb3deec2b" containerName="test-curl-7844-038158"
Sep 10 13:54:30.695266 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.695220    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8acebaa-c492-4a43-b23e-2a9eb3deec2b" containerName="test-curl-7844-038158"
Sep 10 13:54:30.698149 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.698132    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.700801 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.700774    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-579ae1\"/\"openshift-service-ca.crt\""
Sep 10 13:54:30.701881 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.701849    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-579ae1\"/\"default-dockercfg-rjdk6\""
Sep 10 13:54:30.702003 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.701914    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-579ae1\"/\"sub-test-model-579ae1-kserve-self-signed-certs\""
Sep 10 13:54:30.702003 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.701919    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-579ae1\"/\"kube-root-ca.crt\""
Sep 10 13:54:30.712417 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.712393    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"]
Sep 10 13:54:30.832538 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.832499    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psmzz\" (UniqueName: \"kubernetes.io/projected/fee46724-109d-4169-a4af-b4d3190277fd-kube-api-access-psmzz\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.832703 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.832552    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-dshm\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.832703 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.832616    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-tmp-dir\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.832703 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.832644    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-home\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.832703 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.832677    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-model-cache\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.832703 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.832698    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fee46724-109d-4169-a4af-b4d3190277fd-tls-certs\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.933115 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.933076    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-model-cache\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.933115 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.933113    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fee46724-109d-4169-a4af-b4d3190277fd-tls-certs\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.933317 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.933147    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psmzz\" (UniqueName: \"kubernetes.io/projected/fee46724-109d-4169-a4af-b4d3190277fd-kube-api-access-psmzz\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.933317 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.933202    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-dshm\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.933317 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.933256    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-tmp-dir\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.933317 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.933278    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-home\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.933538 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.933491    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-model-cache\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.933608 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.933576    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-home\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.933665 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.933645    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-tmp-dir\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.935513 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.935486    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-dshm\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.935740 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.935723    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fee46724-109d-4169-a4af-b4d3190277fd-tls-certs\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:30.942405 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:30.942381    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psmzz\" (UniqueName: \"kubernetes.io/projected/fee46724-109d-4169-a4af-b4d3190277fd-kube-api-access-psmzz\") pod \"sub-test-model-579ae1-kserve-cb44766c5-rdqzx\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") " pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:31.007161 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:31.007085    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:31.285539 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:31.285403    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"]
Sep 10 13:54:31.288136 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:54:31.288105    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfee46724_109d_4169_a4af_b4d3190277fd.slice/crio-32c35cf5c8a799d914b1d54fafaf868603b94ef66c0f4d493cc673f66aca4769 WatchSource:0}: Error finding container 32c35cf5c8a799d914b1d54fafaf868603b94ef66c0f4d493cc673f66aca4769: Status 404 returned error can't find the container with id 32c35cf5c8a799d914b1d54fafaf868603b94ef66c0f4d493cc673f66aca4769
Sep 10 13:54:31.547439 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:31.547338    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx" event={"ID":"fee46724-109d-4169-a4af-b4d3190277fd","Type":"ContainerStarted","Data":"105120e49dfded7d151e92bf53968a704fc038228dc9e11ae1d3b7be18dbda99"}
Sep 10 13:54:31.547439 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:31.547383    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx" event={"ID":"fee46724-109d-4169-a4af-b4d3190277fd","Type":"ContainerStarted","Data":"32c35cf5c8a799d914b1d54fafaf868603b94ef66c0f4d493cc673f66aca4769"}
Sep 10 13:54:31.571277 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:31.571233    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx" podStartSLOduration=1.571219157 podStartE2EDuration="1.571219157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:54:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:54:31.569212552 +0000 UTC m=+1430.475770597" watchObservedRunningTime="2026-09-10 13:54:31.571219157 +0000 UTC m=+1430.477777240"
Sep 10 13:54:41.007943 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:41.007838    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:41.007943 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:41.007922    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:41.020459 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:41.020431    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:54:41.603961 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:54:41.603931    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:55:07.888156 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.888122    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"]
Sep 10 13:55:07.892522 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.892504    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:07.895664 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.895641    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-d5c012\"/\"default-dockercfg-5rrv9\""
Sep 10 13:55:07.895821 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.895706    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-d5c012\"/\"openshift-service-ca.crt\""
Sep 10 13:55:07.896729 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.896713    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-d5c012\"/\"sub-test-model-d5c012-kserve-self-signed-certs\""
Sep 10 13:55:07.896840 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.896822    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-d5c012\"/\"kube-root-ca.crt\""
Sep 10 13:55:07.907479 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.907457    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"]
Sep 10 13:55:07.953224 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.953196    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqsm9\" (UniqueName: \"kubernetes.io/projected/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-kube-api-access-lqsm9\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:07.953224 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.953227    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-model-cache\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:07.953421 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.953247    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-home\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:07.953421 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.953262    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tmp-dir\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:07.953421 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.953315    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-dshm\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:07.953421 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:07.953418    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tls-certs\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.054412 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.054377    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tls-certs\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.054610 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.054419    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqsm9\" (UniqueName: \"kubernetes.io/projected/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-kube-api-access-lqsm9\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.054610 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.054444    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-model-cache\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.054610 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.054603    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-home\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.054791 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.054636    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tmp-dir\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.054791 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.054666    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-dshm\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.054935 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.054860    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-model-cache\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.054992 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.054979    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-home\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.055034 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.055003    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tmp-dir\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.056832 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.056814    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-dshm\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.056929 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.056912    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tls-certs\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.064158 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.064126    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqsm9\" (UniqueName: \"kubernetes.io/projected/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-kube-api-access-lqsm9\") pod \"sub-test-model-d5c012-kserve-594d9d4857-ksp75\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.202012 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.201985    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:08.480563 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.480536    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"]
Sep 10 13:55:08.482925 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:55:08.482898    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d9aa66d_1a3d_4f71_b85f_993d6d59172a.slice/crio-5f77fc730dc7fc5fe0a7b131d0bbb6ff0a144fd8120e479ae4af42070f0d5b1d WatchSource:0}: Error finding container 5f77fc730dc7fc5fe0a7b131d0bbb6ff0a144fd8120e479ae4af42070f0d5b1d: Status 404 returned error can't find the container with id 5f77fc730dc7fc5fe0a7b131d0bbb6ff0a144fd8120e479ae4af42070f0d5b1d
Sep 10 13:55:08.712751 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.712717    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75" event={"ID":"8d9aa66d-1a3d-4f71-b85f-993d6d59172a","Type":"ContainerStarted","Data":"0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda"}
Sep 10 13:55:08.712751 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.712757    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75" event={"ID":"8d9aa66d-1a3d-4f71-b85f-993d6d59172a","Type":"ContainerStarted","Data":"5f77fc730dc7fc5fe0a7b131d0bbb6ff0a144fd8120e479ae4af42070f0d5b1d"}
Sep 10 13:55:08.734699 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:08.734610    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75" podStartSLOduration=1.734595705 podStartE2EDuration="1.734595705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:55:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:55:08.732843052 +0000 UTC m=+1467.639401085" watchObservedRunningTime="2026-09-10 13:55:08.734595705 +0000 UTC m=+1467.641153736"
Sep 10 13:55:18.202560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:18.202524    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:18.203033 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:18.202612    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:18.215308 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:18.215286    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:18.767943 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:18.767915    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:55:35.581514 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.581477    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"]
Sep 10 13:55:35.584675 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.584658    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:55:35.588030 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.588007    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-fc1974-dockercfg-q75qz\""
Sep 10 13:55:35.588535 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.588518    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-fc1974\""
Sep 10 13:55:35.600588 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.600562    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"]
Sep 10 13:55:35.689198 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.689166    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d21e0568-3c88-4059-af71-bfa9f0d3e671-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c\" (UID: \"d21e0568-3c88-4059-af71-bfa9f0d3e671\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:55:35.689198 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.689201    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98dfg\" (UniqueName: \"kubernetes.io/projected/d21e0568-3c88-4059-af71-bfa9f0d3e671-kube-api-access-98dfg\") pod \"payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c\" (UID: \"d21e0568-3c88-4059-af71-bfa9f0d3e671\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:55:35.790057 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.790021    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d21e0568-3c88-4059-af71-bfa9f0d3e671-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c\" (UID: \"d21e0568-3c88-4059-af71-bfa9f0d3e671\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:55:35.790057 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.790058    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98dfg\" (UniqueName: \"kubernetes.io/projected/d21e0568-3c88-4059-af71-bfa9f0d3e671-kube-api-access-98dfg\") pod \"payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c\" (UID: \"d21e0568-3c88-4059-af71-bfa9f0d3e671\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:55:35.790747 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.790722    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d21e0568-3c88-4059-af71-bfa9f0d3e671-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c\" (UID: \"d21e0568-3c88-4059-af71-bfa9f0d3e671\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:55:35.806085 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.806055    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98dfg\" (UniqueName: \"kubernetes.io/projected/d21e0568-3c88-4059-af71-bfa9f0d3e671-kube-api-access-98dfg\") pod \"payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c\" (UID: \"d21e0568-3c88-4059-af71-bfa9f0d3e671\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:55:35.895921 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:35.895832    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:55:36.189144 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:36.189105    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"]
Sep 10 13:55:36.192225 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:55:36.192200    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21e0568_3c88_4059_af71_bfa9f0d3e671.slice/crio-358590078c59269ebb147fd786f9f89e56220f4b7eeee9c128cb848ab7490cb4 WatchSource:0}: Error finding container 358590078c59269ebb147fd786f9f89e56220f4b7eeee9c128cb848ab7490cb4: Status 404 returned error can't find the container with id 358590078c59269ebb147fd786f9f89e56220f4b7eeee9c128cb848ab7490cb4
Sep 10 13:55:36.842629 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:36.842590    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c" event={"ID":"d21e0568-3c88-4059-af71-bfa9f0d3e671","Type":"ContainerStarted","Data":"102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb"}
Sep 10 13:55:36.843013 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:36.842653    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c" event={"ID":"d21e0568-3c88-4059-af71-bfa9f0d3e671","Type":"ContainerStarted","Data":"358590078c59269ebb147fd786f9f89e56220f4b7eeee9c128cb848ab7490cb4"}
Sep 10 13:55:36.843013 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:36.842708    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:55:36.890480 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:36.890433    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c" podStartSLOduration=1.890417563 podStartE2EDuration="1.890417563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:55:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:55:36.889539212 +0000 UTC m=+1495.796097243" watchObservedRunningTime="2026-09-10 13:55:36.890417563 +0000 UTC m=+1495.796975593"
Sep 10 13:55:41.586354 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:41.586328    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:55:41.589123 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:41.589101    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 13:55:41.713606 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:41.713579    2549 scope.go:117] "RemoveContainer" containerID="361fe5762bdd43a18ea62ae4416730fc6bb3f9170dfed50b15cc5557a9c03cb8"
Sep 10 13:55:47.849054 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:55:47.849020    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:56:00.666075 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:00.666043    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7844-797579"]
Sep 10 13:56:00.669408 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:00.669378    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-797579"
Sep 10 13:56:00.672711 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:00.672689    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-g8rj6\""
Sep 10 13:56:00.684237 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:00.684213    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7844-797579"]
Sep 10 13:56:00.706305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:00.706281    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6zpq\" (UniqueName: \"kubernetes.io/projected/01220cac-f56d-4a2a-897d-0e8cc8744796-kube-api-access-l6zpq\") pod \"test-curl-7844-797579\" (UID: \"01220cac-f56d-4a2a-897d-0e8cc8744796\") " pod="opendatahub/test-curl-7844-797579"
Sep 10 13:56:00.807353 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:00.807321    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6zpq\" (UniqueName: \"kubernetes.io/projected/01220cac-f56d-4a2a-897d-0e8cc8744796-kube-api-access-l6zpq\") pod \"test-curl-7844-797579\" (UID: \"01220cac-f56d-4a2a-897d-0e8cc8744796\") " pod="opendatahub/test-curl-7844-797579"
Sep 10 13:56:00.817983 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:00.817955    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6zpq\" (UniqueName: \"kubernetes.io/projected/01220cac-f56d-4a2a-897d-0e8cc8744796-kube-api-access-l6zpq\") pod \"test-curl-7844-797579\" (UID: \"01220cac-f56d-4a2a-897d-0e8cc8744796\") " pod="opendatahub/test-curl-7844-797579"
Sep 10 13:56:00.978986 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:00.978916    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-797579"
Sep 10 13:56:01.252072 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.251977    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7844-797579"]
Sep 10 13:56:01.254911 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:56:01.254884    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01220cac_f56d_4a2a_897d_0e8cc8744796.slice/crio-d80796b84c9703e5d601c5dd7df873afbdc6a58617ddef4c662527c91b8b6b67 WatchSource:0}: Error finding container d80796b84c9703e5d601c5dd7df873afbdc6a58617ddef4c662527c91b8b6b67: Status 404 returned error can't find the container with id d80796b84c9703e5d601c5dd7df873afbdc6a58617ddef4c662527c91b8b6b67
Sep 10 13:56:01.382475 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.382442    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"]
Sep 10 13:56:01.385644 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.385629    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"
Sep 10 13:56:01.388200 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.388180    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:56:01.388458 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.388443    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 13:56:01.395885 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.395849    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"]
Sep 10 13:56:01.412112 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.412089    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0f1f9df-3752-4cf0-a9e4-50408409c2b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv\" (UID: \"d0f1f9df-3752-4cf0-a9e4-50408409c2b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"
Sep 10 13:56:01.512991 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.512908    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0f1f9df-3752-4cf0-a9e4-50408409c2b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv\" (UID: \"d0f1f9df-3752-4cf0-a9e4-50408409c2b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"
Sep 10 13:56:01.513485 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.513463    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0f1f9df-3752-4cf0-a9e4-50408409c2b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv\" (UID: \"d0f1f9df-3752-4cf0-a9e4-50408409c2b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"
Sep 10 13:56:01.695825 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.695796    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"
Sep 10 13:56:01.947259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.947222    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7844-797579" event={"ID":"01220cac-f56d-4a2a-897d-0e8cc8744796","Type":"ContainerStarted","Data":"f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa"}
Sep 10 13:56:01.947259 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.947275    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7844-797579" event={"ID":"01220cac-f56d-4a2a-897d-0e8cc8744796","Type":"ContainerStarted","Data":"d80796b84c9703e5d601c5dd7df873afbdc6a58617ddef4c662527c91b8b6b67"}
Sep 10 13:56:01.965351 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.965305    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"]
Sep 10 13:56:01.967426 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:56:01.967396    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0f1f9df_3752_4cf0_a9e4_50408409c2b6.slice/crio-750a0e1f98cd20fc9d6f949406fce3a95e5331eb491b02fae898ae6f1cd78d21 WatchSource:0}: Error finding container 750a0e1f98cd20fc9d6f949406fce3a95e5331eb491b02fae898ae6f1cd78d21: Status 404 returned error can't find the container with id 750a0e1f98cd20fc9d6f949406fce3a95e5331eb491b02fae898ae6f1cd78d21
Sep 10 13:56:01.970834 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:01.970798    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7844-797579" podStartSLOduration=1.9707848220000002 podStartE2EDuration="1.970784822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:56:01.969059437 +0000 UTC m=+1520.875617478" watchObservedRunningTime="2026-09-10 13:56:01.970784822 +0000 UTC m=+1520.877342851"
Sep 10 13:56:02.657431 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.657399    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7844-797579"]
Sep 10 13:56:02.660384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.660360    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7844-797579"]
Sep 10 13:56:02.786794 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.786762    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7844-89b1be"]
Sep 10 13:56:02.787167 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.787153    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01220cac-f56d-4a2a-897d-0e8cc8744796" containerName="test-curl-7844-797579"
Sep 10 13:56:02.787210 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.787168    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="01220cac-f56d-4a2a-897d-0e8cc8744796" containerName="test-curl-7844-797579"
Sep 10 13:56:02.787250 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.787241    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="01220cac-f56d-4a2a-897d-0e8cc8744796" containerName="test-curl-7844-797579"
Sep 10 13:56:02.790120 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.790104    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-89b1be"
Sep 10 13:56:02.795810 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.795782    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7844-89b1be"]
Sep 10 13:56:02.824460 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.824393    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm94w\" (UniqueName: \"kubernetes.io/projected/76090be2-890e-419c-939a-582102591772-kube-api-access-gm94w\") pod \"test-curl-7844-89b1be\" (UID: \"76090be2-890e-419c-939a-582102591772\") " pod="opendatahub/test-curl-7844-89b1be"
Sep 10 13:56:02.924819 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.924788    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm94w\" (UniqueName: \"kubernetes.io/projected/76090be2-890e-419c-939a-582102591772-kube-api-access-gm94w\") pod \"test-curl-7844-89b1be\" (UID: \"76090be2-890e-419c-939a-582102591772\") " pod="opendatahub/test-curl-7844-89b1be"
Sep 10 13:56:02.935248 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.935222    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm94w\" (UniqueName: \"kubernetes.io/projected/76090be2-890e-419c-939a-582102591772-kube-api-access-gm94w\") pod \"test-curl-7844-89b1be\" (UID: \"76090be2-890e-419c-939a-582102591772\") " pod="opendatahub/test-curl-7844-89b1be"
Sep 10 13:56:02.953499 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.953476    2549 generic.go:356] "Generic (PLEG): container finished" podID="d0f1f9df-3752-4cf0-a9e4-50408409c2b6" containerID="e550a62fd982e79aa5239fb1984bc1969b7df52738dcb92a42c83f8fcc5b17fd" exitCode=0
Sep 10 13:56:02.953622 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.953529    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv" event={"ID":"d0f1f9df-3752-4cf0-a9e4-50408409c2b6","Type":"ContainerDied","Data":"e550a62fd982e79aa5239fb1984bc1969b7df52738dcb92a42c83f8fcc5b17fd"}
Sep 10 13:56:02.953622 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:02.953576    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv" event={"ID":"d0f1f9df-3752-4cf0-a9e4-50408409c2b6","Type":"ContainerStarted","Data":"750a0e1f98cd20fc9d6f949406fce3a95e5331eb491b02fae898ae6f1cd78d21"}
Sep 10 13:56:02.953701 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:56:02.953685    2549 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7844-797579" container="test-curl-7844-797579"
Sep 10 13:56:03.100881 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:03.100797    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-89b1be"
Sep 10 13:56:03.373130 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:03.373106    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7844-89b1be"]
Sep 10 13:56:03.375050 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:56:03.375021    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76090be2_890e_419c_939a_582102591772.slice/crio-05f108cb07ee2ee97731ff6304013af29676f8e1c58d836ce64008d964208552 WatchSource:0}: Error finding container 05f108cb07ee2ee97731ff6304013af29676f8e1c58d836ce64008d964208552: Status 404 returned error can't find the container with id 05f108cb07ee2ee97731ff6304013af29676f8e1c58d836ce64008d964208552
Sep 10 13:56:03.960206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:03.960116    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7844-89b1be" event={"ID":"76090be2-890e-419c-939a-582102591772","Type":"ContainerStarted","Data":"8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e"}
Sep 10 13:56:03.960206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:03.960159    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7844-89b1be" event={"ID":"76090be2-890e-419c-939a-582102591772","Type":"ContainerStarted","Data":"05f108cb07ee2ee97731ff6304013af29676f8e1c58d836ce64008d964208552"}
Sep 10 13:56:03.960206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:03.960157    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7844-797579" podUID="01220cac-f56d-4a2a-897d-0e8cc8744796" containerName="test-curl-7844-797579" containerID="cri-o://f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa" gracePeriod=2
Sep 10 13:56:03.980327 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:03.980279    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7844-89b1be" podStartSLOduration=1.9802628150000001 podStartE2EDuration="1.980262815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:56:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:56:03.979317386 +0000 UTC m=+1522.885875415" watchObservedRunningTime="2026-09-10 13:56:03.980262815 +0000 UTC m=+1522.886820880"
Sep 10 13:56:04.090305 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:04.090283    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"
Sep 10 13:56:04.134802 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:04.134777    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0f1f9df-3752-4cf0-a9e4-50408409c2b6-maas-api-service-ca\") pod \"d0f1f9df-3752-4cf0-a9e4-50408409c2b6\" (UID: \"d0f1f9df-3752-4cf0-a9e4-50408409c2b6\") "
Sep 10 13:56:04.135130 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:04.135107    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0f1f9df-3752-4cf0-a9e4-50408409c2b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d0f1f9df-3752-4cf0-a9e4-50408409c2b6" (UID: "d0f1f9df-3752-4cf0-a9e4-50408409c2b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:56:04.235858 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:04.235778    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0f1f9df-3752-4cf0-a9e4-50408409c2b6-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:04.630651 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:04.630559    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7844-89b1be"]
Sep 10 13:56:04.633007 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:04.632980    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7844-89b1be"]
Sep 10 13:56:04.966028 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:04.966000    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"
Sep 10 13:56:04.966384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:04.966031    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv" event={"ID":"d0f1f9df-3752-4cf0-a9e4-50408409c2b6","Type":"ContainerDied","Data":"750a0e1f98cd20fc9d6f949406fce3a95e5331eb491b02fae898ae6f1cd78d21"}
Sep 10 13:56:04.966384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:04.966075    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="750a0e1f98cd20fc9d6f949406fce3a95e5331eb491b02fae898ae6f1cd78d21"
Sep 10 13:56:04.966384 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:56:04.966343    2549 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7844-89b1be" container="test-curl-7844-89b1be"
Sep 10 13:56:05.970774 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:05.970741    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7844-89b1be" podUID="76090be2-890e-419c-939a-582102591772" containerName="test-curl-7844-89b1be" containerID="cri-o://8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e" gracePeriod=2
Sep 10 13:56:06.133420 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:06.133395    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-797579"
Sep 10 13:56:06.252970 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:06.252900    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6zpq\" (UniqueName: \"kubernetes.io/projected/01220cac-f56d-4a2a-897d-0e8cc8744796-kube-api-access-l6zpq\") pod \"01220cac-f56d-4a2a-897d-0e8cc8744796\" (UID: \"01220cac-f56d-4a2a-897d-0e8cc8744796\") "
Sep 10 13:56:06.254827 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:06.254801    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01220cac-f56d-4a2a-897d-0e8cc8744796-kube-api-access-l6zpq" (OuterVolumeSpecName: "kube-api-access-l6zpq") pod "01220cac-f56d-4a2a-897d-0e8cc8744796" (UID: "01220cac-f56d-4a2a-897d-0e8cc8744796"). InnerVolumeSpecName "kube-api-access-l6zpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:56:06.353656 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:06.353620    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6zpq\" (UniqueName: \"kubernetes.io/projected/01220cac-f56d-4a2a-897d-0e8cc8744796-kube-api-access-l6zpq\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:06.905303 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:06.905273    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"]
Sep 10 13:56:06.975852 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:06.975823    2549 generic.go:356] "Generic (PLEG): container finished" podID="01220cac-f56d-4a2a-897d-0e8cc8744796" containerID="f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa" exitCode=137
Sep 10 13:56:06.976278 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:06.975881    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-797579"
Sep 10 13:56:06.976278 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:06.975935    2549 scope.go:117] "RemoveContainer" containerID="f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa"
Sep 10 13:56:06.986707 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:56:06.986066    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7844-797579\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="01220cac-f56d-4a2a-897d-0e8cc8744796" pod="opendatahub/test-curl-7844-797579"
Sep 10 13:56:06.986707 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:06.986142    2549 scope.go:117] "RemoveContainer" containerID="f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa"
Sep 10 13:56:06.987287 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:56:06.987262    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa\": container with ID starting with f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa not found: ID does not exist" containerID="f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa"
Sep 10 13:56:06.987382 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:06.987295    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa"} err="failed to get container status \"f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa\": rpc error: code = NotFound desc = could not find container \"f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa\": container with ID starting with f6d738fde5ee72ff306bff13d1a0793e610652dcad4ae477a00a310a3abd47aa not found: ID does not exist"
Sep 10 13:56:07.674275 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:07.674233    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01220cac-f56d-4a2a-897d-0e8cc8744796" path="/var/lib/kubelet/pods/01220cac-f56d-4a2a-897d-0e8cc8744796/volumes"
Sep 10 13:56:07.935976 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:07.932712    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"]
Sep 10 13:56:08.156538 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:08.156513    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-89b1be"
Sep 10 13:56:08.271903 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:08.271813    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm94w\" (UniqueName: \"kubernetes.io/projected/76090be2-890e-419c-939a-582102591772-kube-api-access-gm94w\") pod \"76090be2-890e-419c-939a-582102591772\" (UID: \"76090be2-890e-419c-939a-582102591772\") "
Sep 10 13:56:08.273806 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:08.273769    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76090be2-890e-419c-939a-582102591772-kube-api-access-gm94w" (OuterVolumeSpecName: "kube-api-access-gm94w") pod "76090be2-890e-419c-939a-582102591772" (UID: "76090be2-890e-419c-939a-582102591772"). InnerVolumeSpecName "kube-api-access-gm94w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:56:08.373314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:08.373281    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm94w\" (UniqueName: \"kubernetes.io/projected/76090be2-890e-419c-939a-582102591772-kube-api-access-gm94w\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:08.988831 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:08.988792    2549 generic.go:356] "Generic (PLEG): container finished" podID="76090be2-890e-419c-939a-582102591772" containerID="8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e" exitCode=137
Sep 10 13:56:08.989008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:08.988834    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7844-89b1be"
Sep 10 13:56:08.989008 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:08.988913    2549 scope.go:117] "RemoveContainer" containerID="8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e"
Sep 10 13:56:08.997649 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:08.997634    2549 scope.go:117] "RemoveContainer" containerID="8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e"
Sep 10 13:56:08.997918 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:56:08.997901    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e\": container with ID starting with 8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e not found: ID does not exist" containerID="8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e"
Sep 10 13:56:08.997976 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:08.997928    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e"} err="failed to get container status \"8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e\": rpc error: code = NotFound desc = could not find container \"8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e\": container with ID starting with 8d5fd0d303272960966feed503605979f21e5ed625fc8df8b843ecfd549e1a3e not found: ID does not exist"
Sep 10 13:56:08.999528 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:56:08.999509    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7844-89b1be\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="76090be2-890e-419c-939a-582102591772" pod="opendatahub/test-curl-7844-89b1be"
Sep 10 13:56:09.674643 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:09.674602    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76090be2-890e-419c-939a-582102591772" path="/var/lib/kubelet/pods/76090be2-890e-419c-939a-582102591772/volumes"
Sep 10 13:56:17.748756 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:17.748721    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"]
Sep 10 13:56:17.751735 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:17.751708    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5fa55-zctdv"]
Sep 10 13:56:19.673449 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:19.673417    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0f1f9df-3752-4cf0-a9e4-50408409c2b6" path="/var/lib/kubelet/pods/d0f1f9df-3752-4cf0-a9e4-50408409c2b6/volumes"
Sep 10 13:56:21.924420 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:21.924379    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx" podUID="fee46724-109d-4169-a4af-b4d3190277fd" containerName="main" containerID="cri-o://105120e49dfded7d151e92bf53968a704fc038228dc9e11ae1d3b7be18dbda99" gracePeriod=45
Sep 10 13:56:22.046215 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.046185    2549 generic.go:356] "Generic (PLEG): container finished" podID="fee46724-109d-4169-a4af-b4d3190277fd" containerID="105120e49dfded7d151e92bf53968a704fc038228dc9e11ae1d3b7be18dbda99" exitCode=0
Sep 10 13:56:22.046329 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.046262    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx" event={"ID":"fee46724-109d-4169-a4af-b4d3190277fd","Type":"ContainerDied","Data":"105120e49dfded7d151e92bf53968a704fc038228dc9e11ae1d3b7be18dbda99"}
Sep 10 13:56:22.178633 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.178571    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:56:22.289249 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289220    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-home\") pod \"fee46724-109d-4169-a4af-b4d3190277fd\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") "
Sep 10 13:56:22.289249 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289252    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-dshm\") pod \"fee46724-109d-4169-a4af-b4d3190277fd\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") "
Sep 10 13:56:22.289459 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289301    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fee46724-109d-4169-a4af-b4d3190277fd-tls-certs\") pod \"fee46724-109d-4169-a4af-b4d3190277fd\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") "
Sep 10 13:56:22.289459 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289336    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psmzz\" (UniqueName: \"kubernetes.io/projected/fee46724-109d-4169-a4af-b4d3190277fd-kube-api-access-psmzz\") pod \"fee46724-109d-4169-a4af-b4d3190277fd\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") "
Sep 10 13:56:22.289459 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289351    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-model-cache\") pod \"fee46724-109d-4169-a4af-b4d3190277fd\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") "
Sep 10 13:56:22.289459 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289407    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-tmp-dir\") pod \"fee46724-109d-4169-a4af-b4d3190277fd\" (UID: \"fee46724-109d-4169-a4af-b4d3190277fd\") "
Sep 10 13:56:22.289673 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289487    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-home" (OuterVolumeSpecName: "home") pod "fee46724-109d-4169-a4af-b4d3190277fd" (UID: "fee46724-109d-4169-a4af-b4d3190277fd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:22.289673 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289628    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-model-cache" (OuterVolumeSpecName: "model-cache") pod "fee46724-109d-4169-a4af-b4d3190277fd" (UID: "fee46724-109d-4169-a4af-b4d3190277fd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:22.289768 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289676    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fee46724-109d-4169-a4af-b4d3190277fd" (UID: "fee46724-109d-4169-a4af-b4d3190277fd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:22.289768 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289722    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:22.289768 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.289741    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:22.291501 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.291478    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fee46724-109d-4169-a4af-b4d3190277fd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fee46724-109d-4169-a4af-b4d3190277fd" (UID: "fee46724-109d-4169-a4af-b4d3190277fd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:56:22.291589 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.291502    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fee46724-109d-4169-a4af-b4d3190277fd-kube-api-access-psmzz" (OuterVolumeSpecName: "kube-api-access-psmzz") pod "fee46724-109d-4169-a4af-b4d3190277fd" (UID: "fee46724-109d-4169-a4af-b4d3190277fd"). InnerVolumeSpecName "kube-api-access-psmzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:56:22.291589 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.291555    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-dshm" (OuterVolumeSpecName: "dshm") pod "fee46724-109d-4169-a4af-b4d3190277fd" (UID: "fee46724-109d-4169-a4af-b4d3190277fd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:22.390272 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.390244    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:22.390272 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.390270    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fee46724-109d-4169-a4af-b4d3190277fd-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:22.390272 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.390279    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psmzz\" (UniqueName: \"kubernetes.io/projected/fee46724-109d-4169-a4af-b4d3190277fd-kube-api-access-psmzz\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:22.390470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.390287    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fee46724-109d-4169-a4af-b4d3190277fd-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:22.955983 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:22.955941    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75" podUID="8d9aa66d-1a3d-4f71-b85f-993d6d59172a" containerName="main" containerID="cri-o://0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda" gracePeriod=45
Sep 10 13:56:23.051466 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.051444    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"
Sep 10 13:56:23.051608 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.051494    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx" event={"ID":"fee46724-109d-4169-a4af-b4d3190277fd","Type":"ContainerDied","Data":"32c35cf5c8a799d914b1d54fafaf868603b94ef66c0f4d493cc673f66aca4769"}
Sep 10 13:56:23.051608 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.051542    2549 scope.go:117] "RemoveContainer" containerID="105120e49dfded7d151e92bf53968a704fc038228dc9e11ae1d3b7be18dbda99"
Sep 10 13:56:23.085228 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.085194    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"]
Sep 10 13:56:23.089319 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.089297    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-579ae1/sub-test-model-579ae1-kserve-cb44766c5-rdqzx"]
Sep 10 13:56:23.205171 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.205149    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:56:23.297384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.297314    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-dshm\") pod \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") "
Sep 10 13:56:23.297384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.297345    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tmp-dir\") pod \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") "
Sep 10 13:56:23.297384 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.297376    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tls-certs\") pod \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") "
Sep 10 13:56:23.297638 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.297420    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-home\") pod \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") "
Sep 10 13:56:23.297638 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.297443    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-model-cache\") pod \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") "
Sep 10 13:56:23.297638 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.297481    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqsm9\" (UniqueName: \"kubernetes.io/projected/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-kube-api-access-lqsm9\") pod \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\" (UID: \"8d9aa66d-1a3d-4f71-b85f-993d6d59172a\") "
Sep 10 13:56:23.297638 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.297619    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8d9aa66d-1a3d-4f71-b85f-993d6d59172a" (UID: "8d9aa66d-1a3d-4f71-b85f-993d6d59172a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:23.297829 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.297715    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-home" (OuterVolumeSpecName: "home") pod "8d9aa66d-1a3d-4f71-b85f-993d6d59172a" (UID: "8d9aa66d-1a3d-4f71-b85f-993d6d59172a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:23.297829 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.297731    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-model-cache" (OuterVolumeSpecName: "model-cache") pod "8d9aa66d-1a3d-4f71-b85f-993d6d59172a" (UID: "8d9aa66d-1a3d-4f71-b85f-993d6d59172a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:23.297829 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.297794    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:23.299519 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.299498    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-dshm" (OuterVolumeSpecName: "dshm") pod "8d9aa66d-1a3d-4f71-b85f-993d6d59172a" (UID: "8d9aa66d-1a3d-4f71-b85f-993d6d59172a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:23.299624 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.299532    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d9aa66d-1a3d-4f71-b85f-993d6d59172a" (UID: "8d9aa66d-1a3d-4f71-b85f-993d6d59172a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:56:23.299624 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.299609    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-kube-api-access-lqsm9" (OuterVolumeSpecName: "kube-api-access-lqsm9") pod "8d9aa66d-1a3d-4f71-b85f-993d6d59172a" (UID: "8d9aa66d-1a3d-4f71-b85f-993d6d59172a"). InnerVolumeSpecName "kube-api-access-lqsm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:56:23.398336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.398307    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:23.398336 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.398331    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:23.398505 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.398343    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqsm9\" (UniqueName: \"kubernetes.io/projected/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-kube-api-access-lqsm9\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:23.398505 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.398352    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:23.398505 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.398360    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d9aa66d-1a3d-4f71-b85f-993d6d59172a-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:23.674121 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:23.674090    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fee46724-109d-4169-a4af-b4d3190277fd" path="/var/lib/kubelet/pods/fee46724-109d-4169-a4af-b4d3190277fd/volumes"
Sep 10 13:56:24.057096 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:24.057005    2549 generic.go:356] "Generic (PLEG): container finished" podID="8d9aa66d-1a3d-4f71-b85f-993d6d59172a" containerID="0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda" exitCode=0
Sep 10 13:56:24.057096 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:24.057045    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"
Sep 10 13:56:24.057664 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:24.057093    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75" event={"ID":"8d9aa66d-1a3d-4f71-b85f-993d6d59172a","Type":"ContainerDied","Data":"0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda"}
Sep 10 13:56:24.057664 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:24.057138    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75" event={"ID":"8d9aa66d-1a3d-4f71-b85f-993d6d59172a","Type":"ContainerDied","Data":"5f77fc730dc7fc5fe0a7b131d0bbb6ff0a144fd8120e479ae4af42070f0d5b1d"}
Sep 10 13:56:24.057664 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:24.057164    2549 scope.go:117] "RemoveContainer" containerID="0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda"
Sep 10 13:56:24.065202 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:24.065182    2549 scope.go:117] "RemoveContainer" containerID="0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda"
Sep 10 13:56:24.065437 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:56:24.065418    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda\": container with ID starting with 0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda not found: ID does not exist" containerID="0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda"
Sep 10 13:56:24.065482 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:24.065446    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda"} err="failed to get container status \"0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda\": rpc error: code = NotFound desc = could not find container \"0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda\": container with ID starting with 0da90f68ed71b85d3ad2dd62a8fabebb435e34b27fccbdcfd8698d5b4d8b5fda not found: ID does not exist"
Sep 10 13:56:24.079353 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:24.079318    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"]
Sep 10 13:56:24.086851 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:24.086831    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/sub-test-model-d5c012-kserve-594d9d4857-ksp75"]
Sep 10 13:56:25.673498 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:25.673462    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d9aa66d-1a3d-4f71-b85f-993d6d59172a" path="/var/lib/kubelet/pods/8d9aa66d-1a3d-4f71-b85f-993d6d59172a/volumes"
Sep 10 13:56:26.918995 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.918962    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"]
Sep 10 13:56:26.919340 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919286    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d9aa66d-1a3d-4f71-b85f-993d6d59172a" containerName="main"
Sep 10 13:56:26.919340 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919297    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d9aa66d-1a3d-4f71-b85f-993d6d59172a" containerName="main"
Sep 10 13:56:26.919340 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919310    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fee46724-109d-4169-a4af-b4d3190277fd" containerName="main"
Sep 10 13:56:26.919340 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919315    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="fee46724-109d-4169-a4af-b4d3190277fd" containerName="main"
Sep 10 13:56:26.919340 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919323    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0f1f9df-3752-4cf0-a9e4-50408409c2b6" containerName="revoke-keys"
Sep 10 13:56:26.919340 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919329    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0f1f9df-3752-4cf0-a9e4-50408409c2b6" containerName="revoke-keys"
Sep 10 13:56:26.919340 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919340    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76090be2-890e-419c-939a-582102591772" containerName="test-curl-7844-89b1be"
Sep 10 13:56:26.919557 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919345    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="76090be2-890e-419c-939a-582102591772" containerName="test-curl-7844-89b1be"
Sep 10 13:56:26.919557 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919391    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d9aa66d-1a3d-4f71-b85f-993d6d59172a" containerName="main"
Sep 10 13:56:26.919557 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919397    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0f1f9df-3752-4cf0-a9e4-50408409c2b6" containerName="revoke-keys"
Sep 10 13:56:26.919557 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919404    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="76090be2-890e-419c-939a-582102591772" containerName="test-curl-7844-89b1be"
Sep 10 13:56:26.919557 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.919412    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="fee46724-109d-4169-a4af-b4d3190277fd" containerName="main"
Sep 10 13:56:26.923445 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.923426    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"
Sep 10 13:56:26.926224 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.926199    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:56:26.926327 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.926235    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 13:56:26.934374 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:26.934354    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"]
Sep 10 13:56:27.026393 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:27.026366    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de8125f-dd26-4207-a2eb-f151eb3a74fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-fc1974-r9999\" (UID: \"5de8125f-dd26-4207-a2eb-f151eb3a74fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"
Sep 10 13:56:27.126879 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:27.126826    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de8125f-dd26-4207-a2eb-f151eb3a74fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-fc1974-r9999\" (UID: \"5de8125f-dd26-4207-a2eb-f151eb3a74fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"
Sep 10 13:56:27.127437 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:27.127417    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de8125f-dd26-4207-a2eb-f151eb3a74fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-fc1974-r9999\" (UID: \"5de8125f-dd26-4207-a2eb-f151eb3a74fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"
Sep 10 13:56:27.233247 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:27.233165    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"
Sep 10 13:56:27.505044 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:27.505020    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"]
Sep 10 13:56:27.507115 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:56:27.507091    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de8125f_dd26_4207_a2eb_f151eb3a74fe.slice/crio-a29dda33b598b49c9444f8ffc3299b970ef0fc2ef35b94415385f7cebe635934 WatchSource:0}: Error finding container a29dda33b598b49c9444f8ffc3299b970ef0fc2ef35b94415385f7cebe635934: Status 404 returned error can't find the container with id a29dda33b598b49c9444f8ffc3299b970ef0fc2ef35b94415385f7cebe635934
Sep 10 13:56:28.082980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:28.082950    2549 generic.go:356] "Generic (PLEG): container finished" podID="5de8125f-dd26-4207-a2eb-f151eb3a74fe" containerID="d2f41d9c3ab058209c723c5d9eb93ef13e06790bd04640a353ae8be15bcd29a4" exitCode=0
Sep 10 13:56:28.083322 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:28.083038    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999" event={"ID":"5de8125f-dd26-4207-a2eb-f151eb3a74fe","Type":"ContainerDied","Data":"d2f41d9c3ab058209c723c5d9eb93ef13e06790bd04640a353ae8be15bcd29a4"}
Sep 10 13:56:28.083322 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:28.083075    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999" event={"ID":"5de8125f-dd26-4207-a2eb-f151eb3a74fe","Type":"ContainerStarted","Data":"a29dda33b598b49c9444f8ffc3299b970ef0fc2ef35b94415385f7cebe635934"}
Sep 10 13:56:29.211520 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:29.211492    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"
Sep 10 13:56:29.344797 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:29.344717    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de8125f-dd26-4207-a2eb-f151eb3a74fe-maas-api-service-ca\") pod \"5de8125f-dd26-4207-a2eb-f151eb3a74fe\" (UID: \"5de8125f-dd26-4207-a2eb-f151eb3a74fe\") "
Sep 10 13:56:29.345121 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:29.345094    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5de8125f-dd26-4207-a2eb-f151eb3a74fe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5de8125f-dd26-4207-a2eb-f151eb3a74fe" (UID: "5de8125f-dd26-4207-a2eb-f151eb3a74fe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:56:29.445545 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:29.445517    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de8125f-dd26-4207-a2eb-f151eb3a74fe-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:30.094092 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:30.094015    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"
Sep 10 13:56:30.094092 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:30.094044    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999" event={"ID":"5de8125f-dd26-4207-a2eb-f151eb3a74fe","Type":"ContainerDied","Data":"a29dda33b598b49c9444f8ffc3299b970ef0fc2ef35b94415385f7cebe635934"}
Sep 10 13:56:30.094092 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:30.094090    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a29dda33b598b49c9444f8ffc3299b970ef0fc2ef35b94415385f7cebe635934"
Sep 10 13:56:42.728834 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:42.728800    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"]
Sep 10 13:56:42.729283 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:42.729004    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c" podUID="d21e0568-3c88-4059-af71-bfa9f0d3e671" containerName="payload-pre-processing" containerID="cri-o://102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb" gracePeriod=30
Sep 10 13:56:42.931917 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:42.931853    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"]
Sep 10 13:56:42.934928 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:42.934900    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fc1974-r9999"]
Sep 10 13:56:42.976564 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:42.976536    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:56:43.065269 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.065181    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98dfg\" (UniqueName: \"kubernetes.io/projected/d21e0568-3c88-4059-af71-bfa9f0d3e671-kube-api-access-98dfg\") pod \"d21e0568-3c88-4059-af71-bfa9f0d3e671\" (UID: \"d21e0568-3c88-4059-af71-bfa9f0d3e671\") "
Sep 10 13:56:43.065269 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.065234    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d21e0568-3c88-4059-af71-bfa9f0d3e671-plugins-config-volume\") pod \"d21e0568-3c88-4059-af71-bfa9f0d3e671\" (UID: \"d21e0568-3c88-4059-af71-bfa9f0d3e671\") "
Sep 10 13:56:43.067834 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.065736    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21e0568-3c88-4059-af71-bfa9f0d3e671-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d21e0568-3c88-4059-af71-bfa9f0d3e671" (UID: "d21e0568-3c88-4059-af71-bfa9f0d3e671"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:56:43.071911 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.071855    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21e0568-3c88-4059-af71-bfa9f0d3e671-kube-api-access-98dfg" (OuterVolumeSpecName: "kube-api-access-98dfg") pod "d21e0568-3c88-4059-af71-bfa9f0d3e671" (UID: "d21e0568-3c88-4059-af71-bfa9f0d3e671"). InnerVolumeSpecName "kube-api-access-98dfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:56:43.156666 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.156630    2549 generic.go:356] "Generic (PLEG): container finished" podID="d21e0568-3c88-4059-af71-bfa9f0d3e671" containerID="102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb" exitCode=0
Sep 10 13:56:43.156831 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.156684    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"
Sep 10 13:56:43.156831 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.156715    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c" event={"ID":"d21e0568-3c88-4059-af71-bfa9f0d3e671","Type":"ContainerDied","Data":"102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb"}
Sep 10 13:56:43.156831 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.156787    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c" event={"ID":"d21e0568-3c88-4059-af71-bfa9f0d3e671","Type":"ContainerDied","Data":"358590078c59269ebb147fd786f9f89e56220f4b7eeee9c128cb848ab7490cb4"}
Sep 10 13:56:43.156831 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.156808    2549 scope.go:117] "RemoveContainer" containerID="102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb"
Sep 10 13:56:43.165007 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.164991    2549 scope.go:117] "RemoveContainer" containerID="102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb"
Sep 10 13:56:43.165294 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:56:43.165276    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb\": container with ID starting with 102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb not found: ID does not exist" containerID="102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb"
Sep 10 13:56:43.165366 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.165302    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb"} err="failed to get container status \"102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb\": rpc error: code = NotFound desc = could not find container \"102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb\": container with ID starting with 102e7e0cf4ca4040b511184fe8d60fae326c7bda0f89ddbd4ffe45829eec85bb not found: ID does not exist"
Sep 10 13:56:43.166609 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.166593    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98dfg\" (UniqueName: \"kubernetes.io/projected/d21e0568-3c88-4059-af71-bfa9f0d3e671-kube-api-access-98dfg\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:43.166658 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.166611    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d21e0568-3c88-4059-af71-bfa9f0d3e671-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:43.178048 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.178028    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"]
Sep 10 13:56:43.182448 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.182427    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fc1974-75d8bdb7fd-ptg6c"]
Sep 10 13:56:43.673294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.673260    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5de8125f-dd26-4207-a2eb-f151eb3a74fe" path="/var/lib/kubelet/pods/5de8125f-dd26-4207-a2eb-f151eb3a74fe/volumes"
Sep 10 13:56:43.673612 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:43.673599    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d21e0568-3c88-4059-af71-bfa9f0d3e671" path="/var/lib/kubelet/pods/d21e0568-3c88-4059-af71-bfa9f0d3e671/volumes"
Sep 10 13:56:50.796912 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.796853    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"]
Sep 10 13:56:50.797690 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.797668    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5de8125f-dd26-4207-a2eb-f151eb3a74fe" containerName="revoke-keys"
Sep 10 13:56:50.797773 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.797694    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5de8125f-dd26-4207-a2eb-f151eb3a74fe" containerName="revoke-keys"
Sep 10 13:56:50.797773 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.797724    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21e0568-3c88-4059-af71-bfa9f0d3e671" containerName="payload-pre-processing"
Sep 10 13:56:50.797773 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.797733    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21e0568-3c88-4059-af71-bfa9f0d3e671" containerName="payload-pre-processing"
Sep 10 13:56:50.797950 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.797917    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5de8125f-dd26-4207-a2eb-f151eb3a74fe" containerName="revoke-keys"
Sep 10 13:56:50.797950 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.797930    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21e0568-3c88-4059-af71-bfa9f0d3e671" containerName="payload-pre-processing"
Sep 10 13:56:50.802915 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.802893    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"
Sep 10 13:56:50.805362 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.805340    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:56:50.805476 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.805351    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 13:56:50.810324 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.810129    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"]
Sep 10 13:56:50.833600 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.833571    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6879c497-8b3e-4f4b-b1f4-a3e84c2a4566-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc\" (UID: \"6879c497-8b3e-4f4b-b1f4-a3e84c2a4566\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"
Sep 10 13:56:50.934609 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.934563    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6879c497-8b3e-4f4b-b1f4-a3e84c2a4566-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc\" (UID: \"6879c497-8b3e-4f4b-b1f4-a3e84c2a4566\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"
Sep 10 13:56:50.935237 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:50.935215    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6879c497-8b3e-4f4b-b1f4-a3e84c2a4566-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc\" (UID: \"6879c497-8b3e-4f4b-b1f4-a3e84c2a4566\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"
Sep 10 13:56:51.113954 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:51.113849    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"
Sep 10 13:56:51.390766 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:51.390742    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"]
Sep 10 13:56:51.393536 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:56:51.393509    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6879c497_8b3e_4f4b_b1f4_a3e84c2a4566.slice/crio-df530d4b325ae2a8c5e96ece7621f108e4e6de7ce17445a026d74be5ea0109aa WatchSource:0}: Error finding container df530d4b325ae2a8c5e96ece7621f108e4e6de7ce17445a026d74be5ea0109aa: Status 404 returned error can't find the container with id df530d4b325ae2a8c5e96ece7621f108e4e6de7ce17445a026d74be5ea0109aa
Sep 10 13:56:52.200712 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:52.200675    2549 generic.go:356] "Generic (PLEG): container finished" podID="6879c497-8b3e-4f4b-b1f4-a3e84c2a4566" containerID="4eb651e1e03a5b4dc1be74350834ecb7904f13620622ddf7ed660e0de1835f23" exitCode=0
Sep 10 13:56:52.201077 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:52.200747    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc" event={"ID":"6879c497-8b3e-4f4b-b1f4-a3e84c2a4566","Type":"ContainerDied","Data":"4eb651e1e03a5b4dc1be74350834ecb7904f13620622ddf7ed660e0de1835f23"}
Sep 10 13:56:52.201077 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:52.200791    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc" event={"ID":"6879c497-8b3e-4f4b-b1f4-a3e84c2a4566","Type":"ContainerStarted","Data":"df530d4b325ae2a8c5e96ece7621f108e4e6de7ce17445a026d74be5ea0109aa"}
Sep 10 13:56:53.335165 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:53.335138    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"
Sep 10 13:56:53.458185 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:53.458091    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6879c497-8b3e-4f4b-b1f4-a3e84c2a4566-maas-api-service-ca\") pod \"6879c497-8b3e-4f4b-b1f4-a3e84c2a4566\" (UID: \"6879c497-8b3e-4f4b-b1f4-a3e84c2a4566\") "
Sep 10 13:56:53.458470 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:53.458447    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6879c497-8b3e-4f4b-b1f4-a3e84c2a4566-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6879c497-8b3e-4f4b-b1f4-a3e84c2a4566" (UID: "6879c497-8b3e-4f4b-b1f4-a3e84c2a4566"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:56:53.559165 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:53.559129    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6879c497-8b3e-4f4b-b1f4-a3e84c2a4566-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:56:54.210546 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:54.210519    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"
Sep 10 13:56:54.210729 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:54.210519    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc" event={"ID":"6879c497-8b3e-4f4b-b1f4-a3e84c2a4566","Type":"ContainerDied","Data":"df530d4b325ae2a8c5e96ece7621f108e4e6de7ce17445a026d74be5ea0109aa"}
Sep 10 13:56:54.210729 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:56:54.210644    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df530d4b325ae2a8c5e96ece7621f108e4e6de7ce17445a026d74be5ea0109aa"
Sep 10 13:57:01.540312 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:01.540265    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"]
Sep 10 13:57:01.540725 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:01.540523    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86" podUID="e245b3c7-0ead-4b10-ab64-bedfa8b89a5e" containerName="payload-processing" containerID="cri-o://941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d" gracePeriod=30
Sep 10 13:57:01.785728 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:01.785703    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:57:01.926616 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:01.926584    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-plugins-config-volume\") pod \"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e\" (UID: \"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e\") "
Sep 10 13:57:01.926832 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:01.926680    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9998\" (UniqueName: \"kubernetes.io/projected/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-kube-api-access-k9998\") pod \"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e\" (UID: \"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e\") "
Sep 10 13:57:01.927104 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:01.927079    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e245b3c7-0ead-4b10-ab64-bedfa8b89a5e" (UID: "e245b3c7-0ead-4b10-ab64-bedfa8b89a5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:57:01.928755 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:01.928734    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-kube-api-access-k9998" (OuterVolumeSpecName: "kube-api-access-k9998") pod "e245b3c7-0ead-4b10-ab64-bedfa8b89a5e" (UID: "e245b3c7-0ead-4b10-ab64-bedfa8b89a5e"). InnerVolumeSpecName "kube-api-access-k9998". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:57:01.939414 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:01.939386    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"]
Sep 10 13:57:01.943474 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:01.943451    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3d0caa-k5rsc"]
Sep 10 13:57:02.027493 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.027458    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:57:02.027493 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.027488    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9998\" (UniqueName: \"kubernetes.io/projected/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e-kube-api-access-k9998\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:57:02.248492 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.248402    2549 generic.go:356] "Generic (PLEG): container finished" podID="e245b3c7-0ead-4b10-ab64-bedfa8b89a5e" containerID="941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d" exitCode=0
Sep 10 13:57:02.248492 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.248449    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"
Sep 10 13:57:02.248745 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.248453    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86" event={"ID":"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e","Type":"ContainerDied","Data":"941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d"}
Sep 10 13:57:02.248745 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.248595    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86" event={"ID":"e245b3c7-0ead-4b10-ab64-bedfa8b89a5e","Type":"ContainerDied","Data":"931c130ca3d691374f80cc7af4c8b4a9bf139b7e3541dd811c3e04cdfd6122e3"}
Sep 10 13:57:02.248745 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.248633    2549 scope.go:117] "RemoveContainer" containerID="941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d"
Sep 10 13:57:02.258078 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.258058    2549 scope.go:117] "RemoveContainer" containerID="941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d"
Sep 10 13:57:02.258391 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:57:02.258364    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d\": container with ID starting with 941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d not found: ID does not exist" containerID="941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d"
Sep 10 13:57:02.258450 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.258403    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d"} err="failed to get container status \"941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d\": rpc error: code = NotFound desc = could not find container \"941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d\": container with ID starting with 941d4df08785cfbc619b2b2fcdc59312dbc761b2694f6877314cf926c3e2084d not found: ID does not exist"
Sep 10 13:57:02.285241 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.285207    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"]
Sep 10 13:57:02.291764 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:02.291738    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3d0caa-58bdf89585-w2r86"]
Sep 10 13:57:03.479030 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.478995    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"]
Sep 10 13:57:03.479428 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.479330    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e245b3c7-0ead-4b10-ab64-bedfa8b89a5e" containerName="payload-processing"
Sep 10 13:57:03.479428 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.479342    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e245b3c7-0ead-4b10-ab64-bedfa8b89a5e" containerName="payload-processing"
Sep 10 13:57:03.479428 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.479361    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6879c497-8b3e-4f4b-b1f4-a3e84c2a4566" containerName="revoke-keys"
Sep 10 13:57:03.479428 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.479366    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6879c497-8b3e-4f4b-b1f4-a3e84c2a4566" containerName="revoke-keys"
Sep 10 13:57:03.479428 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.479414    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e245b3c7-0ead-4b10-ab64-bedfa8b89a5e" containerName="payload-processing"
Sep 10 13:57:03.479428 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.479422    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="6879c497-8b3e-4f4b-b1f4-a3e84c2a4566" containerName="revoke-keys"
Sep 10 13:57:03.483408 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.483391    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.490442 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.490421    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-d5c012\"/\"default-dockercfg-5rrv9\""
Sep 10 13:57:03.490767 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.490752    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-d5c012\"/\"openshift-service-ca.crt\""
Sep 10 13:57:03.491411 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.491392    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-d5c012\"/\"kube-root-ca.crt\""
Sep 10 13:57:03.492412 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.492397    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-d5c012\"/\"rate-test-model-d5c012-kserve-self-signed-certs\""
Sep 10 13:57:03.502913 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.502890    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"]
Sep 10 13:57:03.541042 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.541012    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfv6l\" (UniqueName: \"kubernetes.io/projected/d4ae7784-8da4-40f3-abaf-687ae6e974c2-kube-api-access-nfv6l\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.541161 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.541071    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-model-cache\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.541161 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.541116    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-home\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.541234 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.541159    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tls-certs\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.541234 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.541213    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-dshm\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.541301 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.541244    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tmp-dir\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.642076 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.642042    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tls-certs\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.642246 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.642130    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-dshm\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.642246 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.642160    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tmp-dir\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.642246 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.642211    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfv6l\" (UniqueName: \"kubernetes.io/projected/d4ae7784-8da4-40f3-abaf-687ae6e974c2-kube-api-access-nfv6l\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.642416 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.642283    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-model-cache\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.642416 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.642317    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-home\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.642683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.642647    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tmp-dir\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.642802 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.642716    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-model-cache\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.643020 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.642999    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-home\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.644461 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.644437    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-dshm\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.645023 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.644994    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tls-certs\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.657683 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.657661    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfv6l\" (UniqueName: \"kubernetes.io/projected/d4ae7784-8da4-40f3-abaf-687ae6e974c2-kube-api-access-nfv6l\") pod \"rate-test-model-d5c012-kserve-67d6d4c68c-969ss\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") " pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:03.673712 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.673684    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6879c497-8b3e-4f4b-b1f4-a3e84c2a4566" path="/var/lib/kubelet/pods/6879c497-8b3e-4f4b-b1f4-a3e84c2a4566/volumes"
Sep 10 13:57:03.674085 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.674071    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e245b3c7-0ead-4b10-ab64-bedfa8b89a5e" path="/var/lib/kubelet/pods/e245b3c7-0ead-4b10-ab64-bedfa8b89a5e/volumes"
Sep 10 13:57:03.792283 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:03.792195    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:04.072958 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:04.072932    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"]
Sep 10 13:57:04.074229 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:57:04.074206    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4ae7784_8da4_40f3_abaf_687ae6e974c2.slice/crio-08ed2bc1d947932960f911dc2f86c54f8716983ecec7d981db5bb77ca3de0b60 WatchSource:0}: Error finding container 08ed2bc1d947932960f911dc2f86c54f8716983ecec7d981db5bb77ca3de0b60: Status 404 returned error can't find the container with id 08ed2bc1d947932960f911dc2f86c54f8716983ecec7d981db5bb77ca3de0b60
Sep 10 13:57:04.260379 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:04.260337    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss" event={"ID":"d4ae7784-8da4-40f3-abaf-687ae6e974c2","Type":"ContainerStarted","Data":"cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51"}
Sep 10 13:57:04.260379 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:04.260383    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss" event={"ID":"d4ae7784-8da4-40f3-abaf-687ae6e974c2","Type":"ContainerStarted","Data":"08ed2bc1d947932960f911dc2f86c54f8716983ecec7d981db5bb77ca3de0b60"}
Sep 10 13:57:04.312631 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:04.312581    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss" podStartSLOduration=1.31256784 podStartE2EDuration="1.31256784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:57:04.312401224 +0000 UTC m=+1583.218959255" watchObservedRunningTime="2026-09-10 13:57:04.31256784 +0000 UTC m=+1583.219125901"
Sep 10 13:57:10.520025 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:10.519994    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"]
Sep 10 13:57:10.523224 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:10.523207    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"
Sep 10 13:57:10.525790 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:10.525765    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 13:57:10.525937 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:10.525837    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:57:10.532533 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:10.532509    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"]
Sep 10 13:57:10.604399 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:10.604369    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94a70bb-c342-4736-ba5d-4f40f62ef44b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw\" (UID: \"e94a70bb-c342-4736-ba5d-4f40f62ef44b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"
Sep 10 13:57:10.705507 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:10.705474    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94a70bb-c342-4736-ba5d-4f40f62ef44b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw\" (UID: \"e94a70bb-c342-4736-ba5d-4f40f62ef44b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"
Sep 10 13:57:10.706149 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:10.706127    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94a70bb-c342-4736-ba5d-4f40f62ef44b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw\" (UID: \"e94a70bb-c342-4736-ba5d-4f40f62ef44b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"
Sep 10 13:57:10.834418 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:10.834324    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"
Sep 10 13:57:11.107420 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:11.107386    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"]
Sep 10 13:57:11.110041 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:57:11.110011    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94a70bb_c342_4736_ba5d_4f40f62ef44b.slice/crio-f902ce9dd39f4e3ead1de0b55b71f4c86a2bfb4dbfb35a753c37840988a94745 WatchSource:0}: Error finding container f902ce9dd39f4e3ead1de0b55b71f4c86a2bfb4dbfb35a753c37840988a94745: Status 404 returned error can't find the container with id f902ce9dd39f4e3ead1de0b55b71f4c86a2bfb4dbfb35a753c37840988a94745
Sep 10 13:57:11.293012 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:11.292979    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw" event={"ID":"e94a70bb-c342-4736-ba5d-4f40f62ef44b","Type":"ContainerStarted","Data":"f902ce9dd39f4e3ead1de0b55b71f4c86a2bfb4dbfb35a753c37840988a94745"}
Sep 10 13:57:12.299670 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:12.299633    2549 generic.go:356] "Generic (PLEG): container finished" podID="e94a70bb-c342-4736-ba5d-4f40f62ef44b" containerID="4035a1e5c2030cbac68fcad486a27b2039b16526e15e1201bdfda36cae70f292" exitCode=0
Sep 10 13:57:12.300133 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:12.299714    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw" event={"ID":"e94a70bb-c342-4736-ba5d-4f40f62ef44b","Type":"ContainerDied","Data":"4035a1e5c2030cbac68fcad486a27b2039b16526e15e1201bdfda36cae70f292"}
Sep 10 13:57:13.430798 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:13.430772    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"
Sep 10 13:57:13.528613 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:13.528578    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94a70bb-c342-4736-ba5d-4f40f62ef44b-maas-api-service-ca\") pod \"e94a70bb-c342-4736-ba5d-4f40f62ef44b\" (UID: \"e94a70bb-c342-4736-ba5d-4f40f62ef44b\") "
Sep 10 13:57:13.528969 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:13.528942    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e94a70bb-c342-4736-ba5d-4f40f62ef44b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e94a70bb-c342-4736-ba5d-4f40f62ef44b" (UID: "e94a70bb-c342-4736-ba5d-4f40f62ef44b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:57:13.629175 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:13.629103    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94a70bb-c342-4736-ba5d-4f40f62ef44b-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:57:13.793296 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:13.793245    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:13.793296 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:13.793307    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:13.805931 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:13.805899    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:14.309778 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:14.309748    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"
Sep 10 13:57:14.309972 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:14.309786    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw" event={"ID":"e94a70bb-c342-4736-ba5d-4f40f62ef44b","Type":"ContainerDied","Data":"f902ce9dd39f4e3ead1de0b55b71f4c86a2bfb4dbfb35a753c37840988a94745"}
Sep 10 13:57:14.309972 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:14.309834    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f902ce9dd39f4e3ead1de0b55b71f4c86a2bfb4dbfb35a753c37840988a94745"
Sep 10 13:57:14.321489 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:14.321465    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:57:21.561982 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.561947    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"]
Sep 10 13:57:21.562442 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.562206    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x" podUID="7e13f416-6911-4b1a-9966-dc0747c19b59" containerName="payload-processing" containerID="cri-o://da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745" gracePeriod=30
Sep 10 13:57:21.581617 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.581589    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"]
Sep 10 13:57:21.581794 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.581775    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j" podUID="9e7dfc60-9855-4211-9c3e-155cb43ae2ac" containerName="payload-pre-processing" containerID="cri-o://f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820" gracePeriod=30
Sep 10 13:57:21.851314 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.851286    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:57:21.853711 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.853691    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:57:21.902333 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.902294    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e13f416-6911-4b1a-9966-dc0747c19b59-plugins-config-volume\") pod \"7e13f416-6911-4b1a-9966-dc0747c19b59\" (UID: \"7e13f416-6911-4b1a-9966-dc0747c19b59\") "
Sep 10 13:57:21.902635 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.902348    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tt4d\" (UniqueName: \"kubernetes.io/projected/7e13f416-6911-4b1a-9966-dc0747c19b59-kube-api-access-2tt4d\") pod \"7e13f416-6911-4b1a-9966-dc0747c19b59\" (UID: \"7e13f416-6911-4b1a-9966-dc0747c19b59\") "
Sep 10 13:57:21.902635 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.902373    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzgbr\" (UniqueName: \"kubernetes.io/projected/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-kube-api-access-qzgbr\") pod \"9e7dfc60-9855-4211-9c3e-155cb43ae2ac\" (UID: \"9e7dfc60-9855-4211-9c3e-155cb43ae2ac\") "
Sep 10 13:57:21.902635 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.902413    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-plugins-config-volume\") pod \"9e7dfc60-9855-4211-9c3e-155cb43ae2ac\" (UID: \"9e7dfc60-9855-4211-9c3e-155cb43ae2ac\") "
Sep 10 13:57:21.902820 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.902709    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e13f416-6911-4b1a-9966-dc0747c19b59-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e13f416-6911-4b1a-9966-dc0747c19b59" (UID: "7e13f416-6911-4b1a-9966-dc0747c19b59"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:57:21.902960 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.902934    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9e7dfc60-9855-4211-9c3e-155cb43ae2ac" (UID: "9e7dfc60-9855-4211-9c3e-155cb43ae2ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:57:21.904537 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.904514    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e13f416-6911-4b1a-9966-dc0747c19b59-kube-api-access-2tt4d" (OuterVolumeSpecName: "kube-api-access-2tt4d") pod "7e13f416-6911-4b1a-9966-dc0747c19b59" (UID: "7e13f416-6911-4b1a-9966-dc0747c19b59"). InnerVolumeSpecName "kube-api-access-2tt4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:57:21.904599 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:21.904535    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-kube-api-access-qzgbr" (OuterVolumeSpecName: "kube-api-access-qzgbr") pod "9e7dfc60-9855-4211-9c3e-155cb43ae2ac" (UID: "9e7dfc60-9855-4211-9c3e-155cb43ae2ac"). InnerVolumeSpecName "kube-api-access-qzgbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:57:22.003510 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.003476    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:57:22.003510 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.003506    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e13f416-6911-4b1a-9966-dc0747c19b59-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:57:22.003510 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.003515    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tt4d\" (UniqueName: \"kubernetes.io/projected/7e13f416-6911-4b1a-9966-dc0747c19b59-kube-api-access-2tt4d\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:57:22.003750 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.003524    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzgbr\" (UniqueName: \"kubernetes.io/projected/9e7dfc60-9855-4211-9c3e-155cb43ae2ac-kube-api-access-qzgbr\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:57:22.205823 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.205787    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"]
Sep 10 13:57:22.221881 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.221841    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b3723e-n8fdw"]
Sep 10 13:57:22.352181 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.352145    2549 generic.go:356] "Generic (PLEG): container finished" podID="9e7dfc60-9855-4211-9c3e-155cb43ae2ac" containerID="f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820" exitCode=0
Sep 10 13:57:22.352426 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.352194    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"
Sep 10 13:57:22.352426 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.352234    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j" event={"ID":"9e7dfc60-9855-4211-9c3e-155cb43ae2ac","Type":"ContainerDied","Data":"f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820"}
Sep 10 13:57:22.352426 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.352283    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j" event={"ID":"9e7dfc60-9855-4211-9c3e-155cb43ae2ac","Type":"ContainerDied","Data":"864f562b32900fe0ce8644df14512ecba74dca4c0e305af5e39cb3c0996632ae"}
Sep 10 13:57:22.352426 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.352311    2549 scope.go:117] "RemoveContainer" containerID="f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820"
Sep 10 13:57:22.353642 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.353617    2549 generic.go:356] "Generic (PLEG): container finished" podID="7e13f416-6911-4b1a-9966-dc0747c19b59" containerID="da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745" exitCode=0
Sep 10 13:57:22.353736 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.353654    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"
Sep 10 13:57:22.353736 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.353698    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x" event={"ID":"7e13f416-6911-4b1a-9966-dc0747c19b59","Type":"ContainerDied","Data":"da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745"}
Sep 10 13:57:22.353825 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.353749    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x" event={"ID":"7e13f416-6911-4b1a-9966-dc0747c19b59","Type":"ContainerDied","Data":"8420ee974ccea15b333d079d094d8fb507f4f6f759ff94fccba66fb7432d3821"}
Sep 10 13:57:22.361204 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.361101    2549 scope.go:117] "RemoveContainer" containerID="f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820"
Sep 10 13:57:22.361442 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:57:22.361417    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820\": container with ID starting with f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820 not found: ID does not exist" containerID="f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820"
Sep 10 13:57:22.361534 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.361453    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820"} err="failed to get container status \"f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820\": rpc error: code = NotFound desc = could not find container \"f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820\": container with ID starting with f6f48fdf25f5668d1e4ccab63fcf9a631645626928a80a26762bb6c610425820 not found: ID does not exist"
Sep 10 13:57:22.361534 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.361473    2549 scope.go:117] "RemoveContainer" containerID="da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745"
Sep 10 13:57:22.369256 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.369239    2549 scope.go:117] "RemoveContainer" containerID="da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745"
Sep 10 13:57:22.369547 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:57:22.369530    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745\": container with ID starting with da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745 not found: ID does not exist" containerID="da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745"
Sep 10 13:57:22.369596 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.369554    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745"} err="failed to get container status \"da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745\": rpc error: code = NotFound desc = could not find container \"da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745\": container with ID starting with da572608c25df410ec6127e8c66362bfbcbd33360cb1b24fbc18a73f7ccee745 not found: ID does not exist"
Sep 10 13:57:22.436159 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.436056    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"]
Sep 10 13:57:22.437585 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.437551    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b3723e-5cbcdccc7d-24r2j"]
Sep 10 13:57:22.462101 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.462011    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"]
Sep 10 13:57:22.467527 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:22.467501    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b3723e-5b97db79f9-fkg6x"]
Sep 10 13:57:23.674173 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:23.674139    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e13f416-6911-4b1a-9966-dc0747c19b59" path="/var/lib/kubelet/pods/7e13f416-6911-4b1a-9966-dc0747c19b59/volumes"
Sep 10 13:57:23.674550 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:23.674491    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e7dfc60-9855-4211-9c3e-155cb43ae2ac" path="/var/lib/kubelet/pods/9e7dfc60-9855-4211-9c3e-155cb43ae2ac/volumes"
Sep 10 13:57:23.674781 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:23.674768    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e94a70bb-c342-4736-ba5d-4f40f62ef44b" path="/var/lib/kubelet/pods/e94a70bb-c342-4736-ba5d-4f40f62ef44b/volumes"
Sep 10 13:57:57.880647 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:57:57.880613    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"]
Sep 10 13:58:12.900980 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:12.900914    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss" podUID="d4ae7784-8da4-40f3-abaf-687ae6e974c2" containerName="main" containerID="cri-o://cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51" gracePeriod=45
Sep 10 13:58:13.143170 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.143146    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:58:13.234101 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.234071    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-home\") pod \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") "
Sep 10 13:58:13.234294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.234117    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-dshm\") pod \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") "
Sep 10 13:58:13.234294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.234145    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tmp-dir\") pod \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") "
Sep 10 13:58:13.234294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.234168    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tls-certs\") pod \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") "
Sep 10 13:58:13.234294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.234190    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-model-cache\") pod \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") "
Sep 10 13:58:13.234294 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.234216    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfv6l\" (UniqueName: \"kubernetes.io/projected/d4ae7784-8da4-40f3-abaf-687ae6e974c2-kube-api-access-nfv6l\") pod \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\" (UID: \"d4ae7784-8da4-40f3-abaf-687ae6e974c2\") "
Sep 10 13:58:13.234560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.234406    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-home" (OuterVolumeSpecName: "home") pod "d4ae7784-8da4-40f3-abaf-687ae6e974c2" (UID: "d4ae7784-8da4-40f3-abaf-687ae6e974c2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:58:13.234560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.234462    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-model-cache" (OuterVolumeSpecName: "model-cache") pod "d4ae7784-8da4-40f3-abaf-687ae6e974c2" (UID: "d4ae7784-8da4-40f3-abaf-687ae6e974c2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:58:13.234560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.234480    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d4ae7784-8da4-40f3-abaf-687ae6e974c2" (UID: "d4ae7784-8da4-40f3-abaf-687ae6e974c2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:58:13.234560 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.234505    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:58:13.236299 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.236258    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-dshm" (OuterVolumeSpecName: "dshm") pod "d4ae7784-8da4-40f3-abaf-687ae6e974c2" (UID: "d4ae7784-8da4-40f3-abaf-687ae6e974c2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:58:13.236299 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.236286    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d4ae7784-8da4-40f3-abaf-687ae6e974c2" (UID: "d4ae7784-8da4-40f3-abaf-687ae6e974c2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:58:13.236299 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.236290    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4ae7784-8da4-40f3-abaf-687ae6e974c2-kube-api-access-nfv6l" (OuterVolumeSpecName: "kube-api-access-nfv6l") pod "d4ae7784-8da4-40f3-abaf-687ae6e974c2" (UID: "d4ae7784-8da4-40f3-abaf-687ae6e974c2"). InnerVolumeSpecName "kube-api-access-nfv6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:58:13.335654 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.335615    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:58:13.335654 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.335645    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:58:13.335654 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.335655    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ae7784-8da4-40f3-abaf-687ae6e974c2-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:58:13.335654 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.335664    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ae7784-8da4-40f3-abaf-687ae6e974c2-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:58:13.335973 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.335673    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfv6l\" (UniqueName: \"kubernetes.io/projected/d4ae7784-8da4-40f3-abaf-687ae6e974c2-kube-api-access-nfv6l\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 13:58:13.583638 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.583548    2549 generic.go:356] "Generic (PLEG): container finished" podID="d4ae7784-8da4-40f3-abaf-687ae6e974c2" containerID="cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51" exitCode=0
Sep 10 13:58:13.583638 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.583585    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"
Sep 10 13:58:13.583883 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.583634    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss" event={"ID":"d4ae7784-8da4-40f3-abaf-687ae6e974c2","Type":"ContainerDied","Data":"cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51"}
Sep 10 13:58:13.583883 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.583679    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss" event={"ID":"d4ae7784-8da4-40f3-abaf-687ae6e974c2","Type":"ContainerDied","Data":"08ed2bc1d947932960f911dc2f86c54f8716983ecec7d981db5bb77ca3de0b60"}
Sep 10 13:58:13.583883 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.583700    2549 scope.go:117] "RemoveContainer" containerID="cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51"
Sep 10 13:58:13.592609 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.592592    2549 scope.go:117] "RemoveContainer" containerID="cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51"
Sep 10 13:58:13.592896 ip-10-0-137-53 kubenswrapper[2549]: E0910 13:58:13.592860    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51\": container with ID starting with cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51 not found: ID does not exist" containerID="cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51"
Sep 10 13:58:13.592951 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.592906    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51"} err="failed to get container status \"cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51\": rpc error: code = NotFound desc = could not find container \"cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51\": container with ID starting with cd674150cd6ba2ff124be2d8eba79ae9c1965a72beebae02f9bec4c034ee1d51 not found: ID does not exist"
Sep 10 13:58:13.611775 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.611753    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"]
Sep 10 13:58:13.619334 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.619312    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d5c012/rate-test-model-d5c012-kserve-67d6d4c68c-969ss"]
Sep 10 13:58:13.674000 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:13.673972    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4ae7784-8da4-40f3-abaf-687ae6e974c2" path="/var/lib/kubelet/pods/d4ae7784-8da4-40f3-abaf-687ae6e974c2/volumes"
Sep 10 13:58:35.911981 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.911945    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"]
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912268    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e7dfc60-9855-4211-9c3e-155cb43ae2ac" containerName="payload-pre-processing"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912280    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e7dfc60-9855-4211-9c3e-155cb43ae2ac" containerName="payload-pre-processing"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912288    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e13f416-6911-4b1a-9966-dc0747c19b59" containerName="payload-processing"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912293    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e13f416-6911-4b1a-9966-dc0747c19b59" containerName="payload-processing"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912304    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ae7784-8da4-40f3-abaf-687ae6e974c2" containerName="main"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912310    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ae7784-8da4-40f3-abaf-687ae6e974c2" containerName="main"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912319    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94a70bb-c342-4736-ba5d-4f40f62ef44b" containerName="revoke-keys"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912324    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94a70bb-c342-4736-ba5d-4f40f62ef44b" containerName="revoke-keys"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912369    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ae7784-8da4-40f3-abaf-687ae6e974c2" containerName="main"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912377    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e13f416-6911-4b1a-9966-dc0747c19b59" containerName="payload-processing"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912384    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e7dfc60-9855-4211-9c3e-155cb43ae2ac" containerName="payload-pre-processing"
Sep 10 13:58:35.912453 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.912390    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e94a70bb-c342-4736-ba5d-4f40f62ef44b" containerName="revoke-keys"
Sep 10 13:58:35.916786 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.916766    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 13:58:35.919179 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.919155    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-fc6ea5-dockercfg-2b9hb\""
Sep 10 13:58:35.919338 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.919321    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-fc6ea5\""
Sep 10 13:58:35.928228 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.928208    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"]
Sep 10 13:58:35.931213 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.931195    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"]
Sep 10 13:58:35.931351 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.931337    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 13:58:35.949002 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:35.948975    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"]
Sep 10 13:58:36.032025 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.031997    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq44b\" (UniqueName: \"kubernetes.io/projected/8b703e6e-720e-4674-9ab0-e9bd30160569-kube-api-access-sq44b\") pod \"payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr\" (UID: \"8b703e6e-720e-4674-9ab0-e9bd30160569\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 13:58:36.032190 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.032045    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/703a58d9-b132-41fc-a0be-9e7885f09da0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt\" (UID: \"703a58d9-b132-41fc-a0be-9e7885f09da0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 13:58:36.032190 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.032071    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b703e6e-720e-4674-9ab0-e9bd30160569-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr\" (UID: \"8b703e6e-720e-4674-9ab0-e9bd30160569\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 13:58:36.032190 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.032131    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx85l\" (UniqueName: \"kubernetes.io/projected/703a58d9-b132-41fc-a0be-9e7885f09da0-kube-api-access-mx85l\") pod \"payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt\" (UID: \"703a58d9-b132-41fc-a0be-9e7885f09da0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 13:58:36.133478 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.133445    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b703e6e-720e-4674-9ab0-e9bd30160569-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr\" (UID: \"8b703e6e-720e-4674-9ab0-e9bd30160569\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 13:58:36.133623 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.133489    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx85l\" (UniqueName: \"kubernetes.io/projected/703a58d9-b132-41fc-a0be-9e7885f09da0-kube-api-access-mx85l\") pod \"payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt\" (UID: \"703a58d9-b132-41fc-a0be-9e7885f09da0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 13:58:36.133623 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.133556    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq44b\" (UniqueName: \"kubernetes.io/projected/8b703e6e-720e-4674-9ab0-e9bd30160569-kube-api-access-sq44b\") pod \"payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr\" (UID: \"8b703e6e-720e-4674-9ab0-e9bd30160569\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 13:58:36.133623 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.133608    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/703a58d9-b132-41fc-a0be-9e7885f09da0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt\" (UID: \"703a58d9-b132-41fc-a0be-9e7885f09da0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 13:58:36.134386 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.134363    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b703e6e-720e-4674-9ab0-e9bd30160569-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr\" (UID: \"8b703e6e-720e-4674-9ab0-e9bd30160569\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 13:58:36.134448 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.134433    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/703a58d9-b132-41fc-a0be-9e7885f09da0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt\" (UID: \"703a58d9-b132-41fc-a0be-9e7885f09da0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 13:58:36.143928 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.143907    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx85l\" (UniqueName: \"kubernetes.io/projected/703a58d9-b132-41fc-a0be-9e7885f09da0-kube-api-access-mx85l\") pod \"payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt\" (UID: \"703a58d9-b132-41fc-a0be-9e7885f09da0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 13:58:36.144040 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.144011    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq44b\" (UniqueName: \"kubernetes.io/projected/8b703e6e-720e-4674-9ab0-e9bd30160569-kube-api-access-sq44b\") pod \"payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr\" (UID: \"8b703e6e-720e-4674-9ab0-e9bd30160569\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 13:58:36.227028 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.226948    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 13:58:36.243290 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.243269    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 13:58:36.504162 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.504132    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"]
Sep 10 13:58:36.506665 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:58:36.506636    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b703e6e_720e_4674_9ab0_e9bd30160569.slice/crio-080b644b7d206aac8da1278f0d16c49101f6dbaa29d030e22216f12a8b9f56e0 WatchSource:0}: Error finding container 080b644b7d206aac8da1278f0d16c49101f6dbaa29d030e22216f12a8b9f56e0: Status 404 returned error can't find the container with id 080b644b7d206aac8da1278f0d16c49101f6dbaa29d030e22216f12a8b9f56e0
Sep 10 13:58:36.521999 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.521972    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"]
Sep 10 13:58:36.523950 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:58:36.523926    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703a58d9_b132_41fc_a0be_9e7885f09da0.slice/crio-7cf2029653b04abf6c02184829170531a7b020c3148b018765277e2ba069a403 WatchSource:0}: Error finding container 7cf2029653b04abf6c02184829170531a7b020c3148b018765277e2ba069a403: Status 404 returned error can't find the container with id 7cf2029653b04abf6c02184829170531a7b020c3148b018765277e2ba069a403
Sep 10 13:58:36.686027 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.685991    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt" event={"ID":"703a58d9-b132-41fc-a0be-9e7885f09da0","Type":"ContainerStarted","Data":"cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1"}
Sep 10 13:58:36.686223 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.686057    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 13:58:36.686223 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.686080    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt" event={"ID":"703a58d9-b132-41fc-a0be-9e7885f09da0","Type":"ContainerStarted","Data":"7cf2029653b04abf6c02184829170531a7b020c3148b018765277e2ba069a403"}
Sep 10 13:58:36.687622 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.687597    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr" event={"ID":"8b703e6e-720e-4674-9ab0-e9bd30160569","Type":"ContainerStarted","Data":"a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d"}
Sep 10 13:58:36.687746 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.687645    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 13:58:36.687746 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.687663    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr" event={"ID":"8b703e6e-720e-4674-9ab0-e9bd30160569","Type":"ContainerStarted","Data":"080b644b7d206aac8da1278f0d16c49101f6dbaa29d030e22216f12a8b9f56e0"}
Sep 10 13:58:36.708765 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.708705    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt" podStartSLOduration=1.708687618 podStartE2EDuration="1.708687618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:58:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:58:36.706995464 +0000 UTC m=+1675.613553494" watchObservedRunningTime="2026-09-10 13:58:36.708687618 +0000 UTC m=+1675.615245653"
Sep 10 13:58:36.729517 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:36.729467    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr" podStartSLOduration=1.729452365 podStartE2EDuration="1.729452365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:58:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:58:36.727364286 +0000 UTC m=+1675.633922317" watchObservedRunningTime="2026-09-10 13:58:36.729452365 +0000 UTC m=+1675.636010395"
Sep 10 13:58:41.773770 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:41.773739    2549 scope.go:117] "RemoveContainer" containerID="73689f3e15874f86e49425de52429b0d204f167e3c02d61dea966b9a05c894d3"
Sep 10 13:58:47.694037 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:47.694007    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 13:58:47.694429 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:58:47.694349    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 13:59:06.869747 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:06.869670    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt_703a58d9-b132-41fc-a0be-9e7885f09da0/payload-processing/0.log"
Sep 10 13:59:35.301816 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.301783    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"]
Sep 10 13:59:35.305063 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.305046    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.307805 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.307777    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0fe383\"/\"default-dockercfg-f9c9c\""
Sep 10 13:59:35.308901 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.308881    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0fe383\"/\"openshift-service-ca.crt\""
Sep 10 13:59:35.309036 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.308888    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0fe383\"/\"kube-root-ca.crt\""
Sep 10 13:59:35.309036 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.308978    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0fe383\"/\"ipp-route-0fe383-kserve-self-signed-certs\""
Sep 10 13:59:35.315678 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.315653    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"]
Sep 10 13:59:35.436885 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.436832    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmpsg\" (UniqueName: \"kubernetes.io/projected/e50a7c5f-77e1-4f87-82b1-096503fa43d5-kube-api-access-xmpsg\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.437064 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.436900    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tmp-dir\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.437064 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.436937    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-home\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.437064 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.436966    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tls-certs\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.437064 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.436991    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-dshm\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.437206 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.437075    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-model-cache\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.538233 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.538190    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmpsg\" (UniqueName: \"kubernetes.io/projected/e50a7c5f-77e1-4f87-82b1-096503fa43d5-kube-api-access-xmpsg\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.538233 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.538236    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tmp-dir\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.538444 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.538369    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-home\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.538444 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.538436    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tls-certs\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.538524 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.538466    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-dshm\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.538587 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.538539    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-model-cache\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.538815 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.538792    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tmp-dir\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.538815 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.538811    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-home\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.538984 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.538935    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-model-cache\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.540697 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.540675    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-dshm\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.541000 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.540981    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tls-certs\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.548023 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.547998    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmpsg\" (UniqueName: \"kubernetes.io/projected/e50a7c5f-77e1-4f87-82b1-096503fa43d5-kube-api-access-xmpsg\") pod \"ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") " pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.615037 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.614952    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:35.905743 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.905657    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"]
Sep 10 13:59:35.909591 ip-10-0-137-53 kubenswrapper[2549]: W0910 13:59:35.909564    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode50a7c5f_77e1_4f87_82b1_096503fa43d5.slice/crio-1dd6c6de8c61d43b66eb95fddfed626e357214b94d8ac75a92c6e9052724ccd7 WatchSource:0}: Error finding container 1dd6c6de8c61d43b66eb95fddfed626e357214b94d8ac75a92c6e9052724ccd7: Status 404 returned error can't find the container with id 1dd6c6de8c61d43b66eb95fddfed626e357214b94d8ac75a92c6e9052724ccd7
Sep 10 13:59:35.959027 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:35.958998    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq" event={"ID":"e50a7c5f-77e1-4f87-82b1-096503fa43d5","Type":"ContainerStarted","Data":"1dd6c6de8c61d43b66eb95fddfed626e357214b94d8ac75a92c6e9052724ccd7"}
Sep 10 13:59:36.966276 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:36.966238    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq" event={"ID":"e50a7c5f-77e1-4f87-82b1-096503fa43d5","Type":"ContainerStarted","Data":"ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0"}
Sep 10 13:59:45.615564 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:45.615524    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:45.616038 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:45.615602    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:45.627688 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:45.627660    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 13:59:45.649132 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:45.649092    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq" podStartSLOduration=10.649081765 podStartE2EDuration="10.649081765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:59:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:59:36.986117989 +0000 UTC m=+1735.892676021" watchObservedRunningTime="2026-09-10 13:59:45.649081765 +0000 UTC m=+1744.555639795"
Sep 10 13:59:46.019383 ip-10-0-137-53 kubenswrapper[2549]: I0910 13:59:46.019355    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 14:00:00.137098 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.137060    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c"]
Sep 10 14:00:00.140589 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.140554    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7"]
Sep 10 14:00:00.140725 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.140607    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c"
Sep 10 14:00:00.143759 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.143738    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7"
Sep 10 14:00:00.144058 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.144032    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct"]
Sep 10 14:00:00.147477 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.147441    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 14:00:00.147638 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.147622    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct"
Sep 10 14:00:00.151619 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.151600    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c"
Sep 10 14:00:00.154560 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.154539    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c"]
Sep 10 14:00:00.159400 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.159370    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct"]
Sep 10 14:00:00.162148 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.162127    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7"
Sep 10 14:00:00.172107 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.172038    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7"]
Sep 10 14:00:00.200252 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.200142    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct"
Sep 10 14:00:00.439458 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.439391    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c"]
Sep 10 14:00:00.442184 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:00:00.442157    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c8f6c0_3e14_449e_9015_7e889c28c430.slice/crio-f7d6904142ff59a3d43bdd89405d871dc33de0fc51915109df09e987f0ebd5ef WatchSource:0}: Error finding container f7d6904142ff59a3d43bdd89405d871dc33de0fc51915109df09e987f0ebd5ef: Status 404 returned error can't find the container with id f7d6904142ff59a3d43bdd89405d871dc33de0fc51915109df09e987f0ebd5ef
Sep 10 14:00:00.454799 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.454776    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7"]
Sep 10 14:00:00.457409 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:00:00.457324    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda20c5f4e_68e2_4d99_8837_a9b25765b962.slice/crio-134daac94373bd3a8171b9f9a176e2d8b01cd9aa60ca8c10d8848b9c37c26a12 WatchSource:0}: Error finding container 134daac94373bd3a8171b9f9a176e2d8b01cd9aa60ca8c10d8848b9c37c26a12: Status 404 returned error can't find the container with id 134daac94373bd3a8171b9f9a176e2d8b01cd9aa60ca8c10d8848b9c37c26a12
Sep 10 14:00:00.498165 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:00.498068    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct"]
Sep 10 14:00:00.499990 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:00:00.499968    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f27317f_84f7_4c8c_a77a_dd5fde070788.slice/crio-0b6e35fc07972820a5df25b00df89139424228f13f48b101520a392c02888734 WatchSource:0}: Error finding container 0b6e35fc07972820a5df25b00df89139424228f13f48b101520a392c02888734: Status 404 returned error can't find the container with id 0b6e35fc07972820a5df25b00df89139424228f13f48b101520a392c02888734
Sep 10 14:00:01.083807 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:01.083711    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c" event={"ID":"c5c8f6c0-3e14-449e-9015-7e889c28c430","Type":"ContainerStarted","Data":"ef6b235a03657313770771dadce009aa7b4b6b3f6533207e346c058be0bf9251"}
Sep 10 14:00:01.083807 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:01.083797    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c" event={"ID":"c5c8f6c0-3e14-449e-9015-7e889c28c430","Type":"ContainerStarted","Data":"f7d6904142ff59a3d43bdd89405d871dc33de0fc51915109df09e987f0ebd5ef"}
Sep 10 14:00:01.085383 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:01.085352    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7" event={"ID":"a20c5f4e-68e2-4d99-8837-a9b25765b962","Type":"ContainerStarted","Data":"2fc253d223e59306be6652e31d9b79f2670d4a4ee8f0d2fd2f934c330d6845b2"}
Sep 10 14:00:01.085480 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:01.085401    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7" event={"ID":"a20c5f4e-68e2-4d99-8837-a9b25765b962","Type":"ContainerStarted","Data":"134daac94373bd3a8171b9f9a176e2d8b01cd9aa60ca8c10d8848b9c37c26a12"}
Sep 10 14:00:01.086708 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:01.086689    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct" event={"ID":"3f27317f-84f7-4c8c-a77a-dd5fde070788","Type":"ContainerStarted","Data":"e1a7cab725c7525b0a592c110b2a5613a9cdeff9326f308cf444035fc356db74"}
Sep 10 14:00:01.086781 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:01.086725    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct" event={"ID":"3f27317f-84f7-4c8c-a77a-dd5fde070788","Type":"ContainerStarted","Data":"0b6e35fc07972820a5df25b00df89139424228f13f48b101520a392c02888734"}
Sep 10 14:00:01.102716 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:01.102673    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c" podStartSLOduration=1.102660725 podStartE2EDuration="1.102660725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:00:01.101301013 +0000 UTC m=+1760.007859045" watchObservedRunningTime="2026-09-10 14:00:01.102660725 +0000 UTC m=+1760.009218757"
Sep 10 14:00:01.117722 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:01.117683    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7" podStartSLOduration=1.117674129 podStartE2EDuration="1.117674129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:00:01.116496165 +0000 UTC m=+1760.023054474" watchObservedRunningTime="2026-09-10 14:00:01.117674129 +0000 UTC m=+1760.024232161"
Sep 10 14:00:01.134581 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:01.134524    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct" podStartSLOduration=1.134505699 podStartE2EDuration="1.134505699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:00:01.13257936 +0000 UTC m=+1760.039137391" watchObservedRunningTime="2026-09-10 14:00:01.134505699 +0000 UTC m=+1760.041063731"
Sep 10 14:00:02.092609 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:02.092574    2549 generic.go:356] "Generic (PLEG): container finished" podID="c5c8f6c0-3e14-449e-9015-7e889c28c430" containerID="ef6b235a03657313770771dadce009aa7b4b6b3f6533207e346c058be0bf9251" exitCode=0
Sep 10 14:00:02.093028 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:02.092662    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c" event={"ID":"c5c8f6c0-3e14-449e-9015-7e889c28c430","Type":"ContainerDied","Data":"ef6b235a03657313770771dadce009aa7b4b6b3f6533207e346c058be0bf9251"}
Sep 10 14:00:02.094257 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:02.094225    2549 generic.go:356] "Generic (PLEG): container finished" podID="a20c5f4e-68e2-4d99-8837-a9b25765b962" containerID="2fc253d223e59306be6652e31d9b79f2670d4a4ee8f0d2fd2f934c330d6845b2" exitCode=0
Sep 10 14:00:02.094368 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:02.094279    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7" event={"ID":"a20c5f4e-68e2-4d99-8837-a9b25765b962","Type":"ContainerDied","Data":"2fc253d223e59306be6652e31d9b79f2670d4a4ee8f0d2fd2f934c330d6845b2"}
Sep 10 14:00:02.095691 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:02.095666    2549 generic.go:356] "Generic (PLEG): container finished" podID="3f27317f-84f7-4c8c-a77a-dd5fde070788" containerID="e1a7cab725c7525b0a592c110b2a5613a9cdeff9326f308cf444035fc356db74" exitCode=0
Sep 10 14:00:02.095807 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:02.095713    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct" event={"ID":"3f27317f-84f7-4c8c-a77a-dd5fde070788","Type":"ContainerDied","Data":"e1a7cab725c7525b0a592c110b2a5613a9cdeff9326f308cf444035fc356db74"}
Sep 10 14:00:03.220947 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:03.220920    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7"
Sep 10 14:00:03.267963 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:03.267943    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c"
Sep 10 14:00:03.271665 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:03.271647    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct"
Sep 10 14:00:03.398741 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:03.398670    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt_703a58d9-b132-41fc-a0be-9e7885f09da0/payload-processing/0.log"
Sep 10 14:00:04.105978 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:04.105896    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c"
Sep 10 14:00:04.105978 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:04.105897    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817480-srh9c" event={"ID":"c5c8f6c0-3e14-449e-9015-7e889c28c430","Type":"ContainerDied","Data":"f7d6904142ff59a3d43bdd89405d871dc33de0fc51915109df09e987f0ebd5ef"}
Sep 10 14:00:04.105978 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:04.105955    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7d6904142ff59a3d43bdd89405d871dc33de0fc51915109df09e987f0ebd5ef"
Sep 10 14:00:04.107366 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:04.107342    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7"
Sep 10 14:00:04.107485 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:04.107381    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7" event={"ID":"a20c5f4e-68e2-4d99-8837-a9b25765b962","Type":"ContainerDied","Data":"134daac94373bd3a8171b9f9a176e2d8b01cd9aa60ca8c10d8848b9c37c26a12"}
Sep 10 14:00:04.107485 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:04.107419    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="134daac94373bd3a8171b9f9a176e2d8b01cd9aa60ca8c10d8848b9c37c26a12"
Sep 10 14:00:04.108770 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:04.108753    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct"
Sep 10 14:00:04.108770 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:04.108765    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct" event={"ID":"3f27317f-84f7-4c8c-a77a-dd5fde070788","Type":"ContainerDied","Data":"0b6e35fc07972820a5df25b00df89139424228f13f48b101520a392c02888734"}
Sep 10 14:00:04.108951 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:04.108803    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b6e35fc07972820a5df25b00df89139424228f13f48b101520a392c02888734"
Sep 10 14:00:11.969673 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.969641    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"]
Sep 10 14:00:11.970156 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.970109    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5c8f6c0-3e14-449e-9015-7e889c28c430" containerName="cleanup"
Sep 10 14:00:11.970156 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.970127    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5c8f6c0-3e14-449e-9015-7e889c28c430" containerName="cleanup"
Sep 10 14:00:11.970156 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.970141    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a20c5f4e-68e2-4d99-8837-a9b25765b962" containerName="cleanup"
Sep 10 14:00:11.970156 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.970148    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="a20c5f4e-68e2-4d99-8837-a9b25765b962" containerName="cleanup"
Sep 10 14:00:11.970389 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.970175    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f27317f-84f7-4c8c-a77a-dd5fde070788" containerName="cleanup"
Sep 10 14:00:11.970389 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.970184    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f27317f-84f7-4c8c-a77a-dd5fde070788" containerName="cleanup"
Sep 10 14:00:11.970389 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.970264    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5c8f6c0-3e14-449e-9015-7e889c28c430" containerName="cleanup"
Sep 10 14:00:11.970389 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.970276    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f27317f-84f7-4c8c-a77a-dd5fde070788" containerName="cleanup"
Sep 10 14:00:11.970389 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.970284    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="a20c5f4e-68e2-4d99-8837-a9b25765b962" containerName="cleanup"
Sep 10 14:00:11.973120 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.973099    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:11.975511 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.975491    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-ae1e0d\""
Sep 10 14:00:11.988377 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:11.988356    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"]
Sep 10 14:00:12.039387 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:12.039359    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc\" (UID: \"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:12.039545 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:12.039404    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpnc6\" (UniqueName: \"kubernetes.io/projected/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-kube-api-access-lpnc6\") pod \"payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc\" (UID: \"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:12.140807 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:12.140771    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc\" (UID: \"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:12.140977 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:12.140822    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpnc6\" (UniqueName: \"kubernetes.io/projected/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-kube-api-access-lpnc6\") pod \"payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc\" (UID: \"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:12.141483 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:12.141462    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc\" (UID: \"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:12.152156 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:12.152132    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpnc6\" (UniqueName: \"kubernetes.io/projected/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-kube-api-access-lpnc6\") pod \"payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc\" (UID: \"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:12.282787 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:12.282683    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:12.576735 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:12.576708    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"]
Sep 10 14:00:12.579370 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:00:12.579339    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a3b7cb_3d9b_4edf_a68a_9b5e47549237.slice/crio-47ed6c4c1db2da89bd3a5c96af4064241f236a63062bedcbee5268bc46206a80 WatchSource:0}: Error finding container 47ed6c4c1db2da89bd3a5c96af4064241f236a63062bedcbee5268bc46206a80: Status 404 returned error can't find the container with id 47ed6c4c1db2da89bd3a5c96af4064241f236a63062bedcbee5268bc46206a80
Sep 10 14:00:13.152832 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:13.152793    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc" event={"ID":"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237","Type":"ContainerStarted","Data":"551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7"}
Sep 10 14:00:13.153246 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:13.152852    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc" event={"ID":"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237","Type":"ContainerStarted","Data":"47ed6c4c1db2da89bd3a5c96af4064241f236a63062bedcbee5268bc46206a80"}
Sep 10 14:00:13.153246 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:13.152895    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:13.175978 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:13.175925    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc" podStartSLOduration=2.175908928 podStartE2EDuration="2.175908928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:00:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:00:13.173539124 +0000 UTC m=+1772.080097155" watchObservedRunningTime="2026-09-10 14:00:13.175908928 +0000 UTC m=+1772.082466960"
Sep 10 14:00:24.160657 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:24.160626    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:37.833335 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:37.833249    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"]
Sep 10 14:00:37.838548 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:37.837996    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"
Sep 10 14:00:37.842493 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:37.842466    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 14:00:37.842652 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:37.842516    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:00:37.849115 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:37.849069    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"]
Sep 10 14:00:37.967893 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:37.967830    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cdc064-c340-4643-821f-1669cce986e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc\" (UID: \"25cdc064-c340-4643-821f-1669cce986e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"
Sep 10 14:00:38.068686 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:38.068645    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cdc064-c340-4643-821f-1669cce986e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc\" (UID: \"25cdc064-c340-4643-821f-1669cce986e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"
Sep 10 14:00:38.069412 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:38.069387    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cdc064-c340-4643-821f-1669cce986e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc\" (UID: \"25cdc064-c340-4643-821f-1669cce986e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"
Sep 10 14:00:38.151836 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:38.151731    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"
Sep 10 14:00:38.431330 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:38.431293    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"]
Sep 10 14:00:38.434838 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:00:38.434810    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25cdc064_c340_4643_821f_1669cce986e3.slice/crio-af3ff322c370ed8c97eebc30f28338cf35f80e8705d61d1b603c1dc81330007a WatchSource:0}: Error finding container af3ff322c370ed8c97eebc30f28338cf35f80e8705d61d1b603c1dc81330007a: Status 404 returned error can't find the container with id af3ff322c370ed8c97eebc30f28338cf35f80e8705d61d1b603c1dc81330007a
Sep 10 14:00:39.274044 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:39.274008    2549 generic.go:356] "Generic (PLEG): container finished" podID="25cdc064-c340-4643-821f-1669cce986e3" containerID="8cfd22b2a38be6d51e7e6278048a43c5d8f7015d2eb5a6bfa0da7445b89f6f89" exitCode=0
Sep 10 14:00:39.274462 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:39.274104    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc" event={"ID":"25cdc064-c340-4643-821f-1669cce986e3","Type":"ContainerDied","Data":"8cfd22b2a38be6d51e7e6278048a43c5d8f7015d2eb5a6bfa0da7445b89f6f89"}
Sep 10 14:00:39.274462 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:39.274158    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc" event={"ID":"25cdc064-c340-4643-821f-1669cce986e3","Type":"ContainerStarted","Data":"af3ff322c370ed8c97eebc30f28338cf35f80e8705d61d1b603c1dc81330007a"}
Sep 10 14:00:40.412030 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:40.412002    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"
Sep 10 14:00:40.588638 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:40.588549    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cdc064-c340-4643-821f-1669cce986e3-maas-api-service-ca\") pod \"25cdc064-c340-4643-821f-1669cce986e3\" (UID: \"25cdc064-c340-4643-821f-1669cce986e3\") "
Sep 10 14:00:40.588948 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:40.588926    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25cdc064-c340-4643-821f-1669cce986e3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "25cdc064-c340-4643-821f-1669cce986e3" (UID: "25cdc064-c340-4643-821f-1669cce986e3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:00:40.689890 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:40.689844    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cdc064-c340-4643-821f-1669cce986e3-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:00:41.285539 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:41.285509    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"
Sep 10 14:00:41.285539 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:41.285522    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc" event={"ID":"25cdc064-c340-4643-821f-1669cce986e3","Type":"ContainerDied","Data":"af3ff322c370ed8c97eebc30f28338cf35f80e8705d61d1b603c1dc81330007a"}
Sep 10 14:00:41.285764 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:41.285571    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af3ff322c370ed8c97eebc30f28338cf35f80e8705d61d1b603c1dc81330007a"
Sep 10 14:00:41.616342 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:41.616249    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 14:00:41.625135 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:41.625102    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 14:00:49.416645 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.416604    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"]
Sep 10 14:00:49.417179 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.416835    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc" podUID="f0a3b7cb-3d9b-4edf-a68a-9b5e47549237" containerName="payload-processing" containerID="cri-o://551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7" gracePeriod=30
Sep 10 14:00:49.656115 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.656086    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"]
Sep 10 14:00:49.660278 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.660250    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ae1e0d-9vstc"]
Sep 10 14:00:49.674370 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.674302    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25cdc064-c340-4643-821f-1669cce986e3" path="/var/lib/kubelet/pods/25cdc064-c340-4643-821f-1669cce986e3/volumes"
Sep 10 14:00:49.674498 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.674373    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:49.764167 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.764130    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-plugins-config-volume\") pod \"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237\" (UID: \"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237\") "
Sep 10 14:00:49.764167 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.764173    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpnc6\" (UniqueName: \"kubernetes.io/projected/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-kube-api-access-lpnc6\") pod \"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237\" (UID: \"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237\") "
Sep 10 14:00:49.764550 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.764525    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0a3b7cb-3d9b-4edf-a68a-9b5e47549237" (UID: "f0a3b7cb-3d9b-4edf-a68a-9b5e47549237"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:00:49.766441 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.766418    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-kube-api-access-lpnc6" (OuterVolumeSpecName: "kube-api-access-lpnc6") pod "f0a3b7cb-3d9b-4edf-a68a-9b5e47549237" (UID: "f0a3b7cb-3d9b-4edf-a68a-9b5e47549237"). InnerVolumeSpecName "kube-api-access-lpnc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:00:49.865586 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.865548    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:00:49.865586 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:49.865583    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpnc6\" (UniqueName: \"kubernetes.io/projected/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237-kube-api-access-lpnc6\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:00:50.332680 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:50.332588    2549 generic.go:356] "Generic (PLEG): container finished" podID="f0a3b7cb-3d9b-4edf-a68a-9b5e47549237" containerID="551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7" exitCode=0
Sep 10 14:00:50.332680 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:50.332639    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"
Sep 10 14:00:50.332680 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:50.332663    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc" event={"ID":"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237","Type":"ContainerDied","Data":"551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7"}
Sep 10 14:00:50.333106 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:50.332707    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc" event={"ID":"f0a3b7cb-3d9b-4edf-a68a-9b5e47549237","Type":"ContainerDied","Data":"47ed6c4c1db2da89bd3a5c96af4064241f236a63062bedcbee5268bc46206a80"}
Sep 10 14:00:50.333106 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:50.332729    2549 scope.go:117] "RemoveContainer" containerID="551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7"
Sep 10 14:00:50.342196 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:50.342174    2549 scope.go:117] "RemoveContainer" containerID="551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7"
Sep 10 14:00:50.342500 ip-10-0-137-53 kubenswrapper[2549]: E0910 14:00:50.342482    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7\": container with ID starting with 551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7 not found: ID does not exist" containerID="551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7"
Sep 10 14:00:50.342581 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:50.342507    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7"} err="failed to get container status \"551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7\": rpc error: code = NotFound desc = could not find container \"551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7\": container with ID starting with 551e74b40eaf36f70f0b3b8d6f6526b280e01836ded256b77ca8f2d97d7138f7 not found: ID does not exist"
Sep 10 14:00:50.356159 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:50.356126    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"]
Sep 10 14:00:50.362241 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:50.362210    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ae1e0d-c65b467c9-6wtqc"]
Sep 10 14:00:51.673761 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:00:51.673723    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0a3b7cb-3d9b-4edf-a68a-9b5e47549237" path="/var/lib/kubelet/pods/f0a3b7cb-3d9b-4edf-a68a-9b5e47549237/volumes"
Sep 10 14:01:01.417249 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.417211    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"]
Sep 10 14:01:01.417636 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.417558    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25cdc064-c340-4643-821f-1669cce986e3" containerName="revoke-keys"
Sep 10 14:01:01.417636 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.417570    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="25cdc064-c340-4643-821f-1669cce986e3" containerName="revoke-keys"
Sep 10 14:01:01.417636 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.417584    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0a3b7cb-3d9b-4edf-a68a-9b5e47549237" containerName="payload-processing"
Sep 10 14:01:01.417636 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.417590    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0a3b7cb-3d9b-4edf-a68a-9b5e47549237" containerName="payload-processing"
Sep 10 14:01:01.417767 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.417641    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0a3b7cb-3d9b-4edf-a68a-9b5e47549237" containerName="payload-processing"
Sep 10 14:01:01.417767 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.417652    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="25cdc064-c340-4643-821f-1669cce986e3" containerName="revoke-keys"
Sep 10 14:01:01.422034 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.422015    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"
Sep 10 14:01:01.424837 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.424810    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 14:01:01.424837 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.424815    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:01:01.430770 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.430709    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"]
Sep 10 14:01:01.471327 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.471290    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f111f4be-36dd-46ab-b1c6-2b85b90b7002-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7\" (UID: \"f111f4be-36dd-46ab-b1c6-2b85b90b7002\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"
Sep 10 14:01:01.572050 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.572012    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f111f4be-36dd-46ab-b1c6-2b85b90b7002-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7\" (UID: \"f111f4be-36dd-46ab-b1c6-2b85b90b7002\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"
Sep 10 14:01:01.572639 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.572618    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f111f4be-36dd-46ab-b1c6-2b85b90b7002-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7\" (UID: \"f111f4be-36dd-46ab-b1c6-2b85b90b7002\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"
Sep 10 14:01:01.733554 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:01.733470    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"
Sep 10 14:01:02.006528 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:02.006356    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"]
Sep 10 14:01:02.009107 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:01:02.009079    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf111f4be_36dd_46ab_b1c6_2b85b90b7002.slice/crio-8d9d416e70bd5923c02071877de7c468e97c1d1d81499cd97b31f44059da70e6 WatchSource:0}: Error finding container 8d9d416e70bd5923c02071877de7c468e97c1d1d81499cd97b31f44059da70e6: Status 404 returned error can't find the container with id 8d9d416e70bd5923c02071877de7c468e97c1d1d81499cd97b31f44059da70e6
Sep 10 14:01:02.389875 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:02.389831    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7" event={"ID":"f111f4be-36dd-46ab-b1c6-2b85b90b7002","Type":"ContainerStarted","Data":"aa4b0198bf4842d479eb9a49278f86b9e2d8f392c79fa3ce45abf372b9f35afb"}
Sep 10 14:01:02.390038 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:02.389900    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7" event={"ID":"f111f4be-36dd-46ab-b1c6-2b85b90b7002","Type":"ContainerStarted","Data":"8d9d416e70bd5923c02071877de7c468e97c1d1d81499cd97b31f44059da70e6"}
Sep 10 14:01:02.409459 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:02.409412    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7" podStartSLOduration=1.409397669 podStartE2EDuration="1.409397669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:01:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:01:02.40756426 +0000 UTC m=+1821.314122291" watchObservedRunningTime="2026-09-10 14:01:02.409397669 +0000 UTC m=+1821.315955744"
Sep 10 14:01:03.395836 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:03.395804    2549 generic.go:356] "Generic (PLEG): container finished" podID="f111f4be-36dd-46ab-b1c6-2b85b90b7002" containerID="aa4b0198bf4842d479eb9a49278f86b9e2d8f392c79fa3ce45abf372b9f35afb" exitCode=0
Sep 10 14:01:03.396233 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:03.395890    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7" event={"ID":"f111f4be-36dd-46ab-b1c6-2b85b90b7002","Type":"ContainerDied","Data":"aa4b0198bf4842d479eb9a49278f86b9e2d8f392c79fa3ce45abf372b9f35afb"}
Sep 10 14:01:04.527944 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:04.527914    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"
Sep 10 14:01:04.598087 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:04.598058    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f111f4be-36dd-46ab-b1c6-2b85b90b7002-maas-api-service-ca\") pod \"f111f4be-36dd-46ab-b1c6-2b85b90b7002\" (UID: \"f111f4be-36dd-46ab-b1c6-2b85b90b7002\") "
Sep 10 14:01:04.598405 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:04.598381    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f111f4be-36dd-46ab-b1c6-2b85b90b7002-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f111f4be-36dd-46ab-b1c6-2b85b90b7002" (UID: "f111f4be-36dd-46ab-b1c6-2b85b90b7002"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:01:04.699595 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:04.699568    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f111f4be-36dd-46ab-b1c6-2b85b90b7002-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:01:05.405776 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:05.405741    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7" event={"ID":"f111f4be-36dd-46ab-b1c6-2b85b90b7002","Type":"ContainerDied","Data":"8d9d416e70bd5923c02071877de7c468e97c1d1d81499cd97b31f44059da70e6"}
Sep 10 14:01:05.405776 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:05.405768    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"
Sep 10 14:01:05.406019 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:05.405794    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d9d416e70bd5923c02071877de7c468e97c1d1d81499cd97b31f44059da70e6"
Sep 10 14:01:17.311284 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:17.311248    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct"]
Sep 10 14:01:17.313610 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:17.313588    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0fe383-29817480-pcvct"]
Sep 10 14:01:17.673238 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:17.673206    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f27317f-84f7-4c8c-a77a-dd5fde070788" path="/var/lib/kubelet/pods/3f27317f-84f7-4c8c-a77a-dd5fde070788/volumes"
Sep 10 14:01:17.705602 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:17.705572    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"]
Sep 10 14:01:17.717480 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:17.717448    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0fe383-djvn7"]
Sep 10 14:01:19.673521 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:19.673489    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f111f4be-36dd-46ab-b1c6-2b85b90b7002" path="/var/lib/kubelet/pods/f111f4be-36dd-46ab-b1c6-2b85b90b7002/volumes"
Sep 10 14:01:23.105159 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:23.105122    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"]
Sep 10 14:01:38.125396 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.125354    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq" podUID="e50a7c5f-77e1-4f87-82b1-096503fa43d5" containerName="main" containerID="cri-o://ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0" gracePeriod=45
Sep 10 14:01:38.362547 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.362525    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 14:01:38.382600 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.382546    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmpsg\" (UniqueName: \"kubernetes.io/projected/e50a7c5f-77e1-4f87-82b1-096503fa43d5-kube-api-access-xmpsg\") pod \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") "
Sep 10 14:01:38.382697 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.382614    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-model-cache\") pod \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") "
Sep 10 14:01:38.382697 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.382650    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-dshm\") pod \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") "
Sep 10 14:01:38.382697 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.382675    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tls-certs\") pod \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") "
Sep 10 14:01:38.382697 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.382693    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tmp-dir\") pod \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") "
Sep 10 14:01:38.382888 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.382713    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-home\") pod \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\" (UID: \"e50a7c5f-77e1-4f87-82b1-096503fa43d5\") "
Sep 10 14:01:38.382950 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.382888    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-model-cache" (OuterVolumeSpecName: "model-cache") pod "e50a7c5f-77e1-4f87-82b1-096503fa43d5" (UID: "e50a7c5f-77e1-4f87-82b1-096503fa43d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:01:38.383015 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.382959    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e50a7c5f-77e1-4f87-82b1-096503fa43d5" (UID: "e50a7c5f-77e1-4f87-82b1-096503fa43d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:01:38.383015 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.382993    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:01:38.383124 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.383079    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-home" (OuterVolumeSpecName: "home") pod "e50a7c5f-77e1-4f87-82b1-096503fa43d5" (UID: "e50a7c5f-77e1-4f87-82b1-096503fa43d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:01:38.384820 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.384793    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50a7c5f-77e1-4f87-82b1-096503fa43d5-kube-api-access-xmpsg" (OuterVolumeSpecName: "kube-api-access-xmpsg") pod "e50a7c5f-77e1-4f87-82b1-096503fa43d5" (UID: "e50a7c5f-77e1-4f87-82b1-096503fa43d5"). InnerVolumeSpecName "kube-api-access-xmpsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:01:38.384820 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.384804    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-dshm" (OuterVolumeSpecName: "dshm") pod "e50a7c5f-77e1-4f87-82b1-096503fa43d5" (UID: "e50a7c5f-77e1-4f87-82b1-096503fa43d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:01:38.384997 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.384805    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e50a7c5f-77e1-4f87-82b1-096503fa43d5" (UID: "e50a7c5f-77e1-4f87-82b1-096503fa43d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:01:38.483824 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.483792    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmpsg\" (UniqueName: \"kubernetes.io/projected/e50a7c5f-77e1-4f87-82b1-096503fa43d5-kube-api-access-xmpsg\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:01:38.483824 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.483819    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:01:38.483824 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.483829    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:01:38.484067 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.483844    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:01:38.484067 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.483853    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e50a7c5f-77e1-4f87-82b1-096503fa43d5-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:01:38.562266 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.562225    2549 generic.go:356] "Generic (PLEG): container finished" podID="e50a7c5f-77e1-4f87-82b1-096503fa43d5" containerID="ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0" exitCode=0
Sep 10 14:01:38.562388 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.562274    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"
Sep 10 14:01:38.562388 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.562292    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq" event={"ID":"e50a7c5f-77e1-4f87-82b1-096503fa43d5","Type":"ContainerDied","Data":"ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0"}
Sep 10 14:01:38.562388 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.562331    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq" event={"ID":"e50a7c5f-77e1-4f87-82b1-096503fa43d5","Type":"ContainerDied","Data":"1dd6c6de8c61d43b66eb95fddfed626e357214b94d8ac75a92c6e9052724ccd7"}
Sep 10 14:01:38.562388 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.562352    2549 scope.go:117] "RemoveContainer" containerID="ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0"
Sep 10 14:01:38.571261 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.571239    2549 scope.go:117] "RemoveContainer" containerID="ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0"
Sep 10 14:01:38.571516 ip-10-0-137-53 kubenswrapper[2549]: E0910 14:01:38.571495    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0\": container with ID starting with ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0 not found: ID does not exist" containerID="ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0"
Sep 10 14:01:38.571578 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.571529    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0"} err="failed to get container status \"ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0\": rpc error: code = NotFound desc = could not find container \"ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0\": container with ID starting with ea2018e6815f12014aa0d7b00a24a54d47caeaba41663febfcbc45b5f221dca0 not found: ID does not exist"
Sep 10 14:01:38.589123 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.589099    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"]
Sep 10 14:01:38.605710 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:38.605678    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-0fe383/ipp-route-0fe383-kserve-6df7bcc5b7-2v6fq"]
Sep 10 14:01:39.674085 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:01:39.674050    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e50a7c5f-77e1-4f87-82b1-096503fa43d5" path="/var/lib/kubelet/pods/e50a7c5f-77e1-4f87-82b1-096503fa43d5/volumes"
Sep 10 14:02:16.365709 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.365618    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"]
Sep 10 14:02:16.366232 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.365969    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f111f4be-36dd-46ab-b1c6-2b85b90b7002" containerName="revoke-keys"
Sep 10 14:02:16.366232 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.365981    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="f111f4be-36dd-46ab-b1c6-2b85b90b7002" containerName="revoke-keys"
Sep 10 14:02:16.366232 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.365991    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e50a7c5f-77e1-4f87-82b1-096503fa43d5" containerName="main"
Sep 10 14:02:16.366232 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.365996    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e50a7c5f-77e1-4f87-82b1-096503fa43d5" containerName="main"
Sep 10 14:02:16.366232 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.366054    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="f111f4be-36dd-46ab-b1c6-2b85b90b7002" containerName="revoke-keys"
Sep 10 14:02:16.366232 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.366065    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e50a7c5f-77e1-4f87-82b1-096503fa43d5" containerName="main"
Sep 10 14:02:16.368921 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.368902    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"
Sep 10 14:02:16.371662 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.371639    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 14:02:16.372020 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.372006    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:02:16.380568 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.380541    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"]
Sep 10 14:02:16.510593 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.510542    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72282fee-4db2-4178-94cc-2a200921272c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh\" (UID: \"72282fee-4db2-4178-94cc-2a200921272c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"
Sep 10 14:02:16.611549 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.611507    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72282fee-4db2-4178-94cc-2a200921272c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh\" (UID: \"72282fee-4db2-4178-94cc-2a200921272c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"
Sep 10 14:02:16.612211 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.612187    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72282fee-4db2-4178-94cc-2a200921272c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh\" (UID: \"72282fee-4db2-4178-94cc-2a200921272c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"
Sep 10 14:02:16.680002 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.679964    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"
Sep 10 14:02:16.953537 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:16.953508    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"]
Sep 10 14:02:16.956388 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:02:16.956357    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72282fee_4db2_4178_94cc_2a200921272c.slice/crio-e17551ddfca8f5afe89f2871510bcbfc28cdb05500d27ed2ff23159e269a2dfd WatchSource:0}: Error finding container e17551ddfca8f5afe89f2871510bcbfc28cdb05500d27ed2ff23159e269a2dfd: Status 404 returned error can't find the container with id e17551ddfca8f5afe89f2871510bcbfc28cdb05500d27ed2ff23159e269a2dfd
Sep 10 14:02:17.750591 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:17.750547    2549 generic.go:356] "Generic (PLEG): container finished" podID="72282fee-4db2-4178-94cc-2a200921272c" containerID="714c3b9cbeb932de7ad22f5a97bef3d2d1fc3b31673cc9934e59625610d45f58" exitCode=0
Sep 10 14:02:17.751175 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:17.750636    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh" event={"ID":"72282fee-4db2-4178-94cc-2a200921272c","Type":"ContainerDied","Data":"714c3b9cbeb932de7ad22f5a97bef3d2d1fc3b31673cc9934e59625610d45f58"}
Sep 10 14:02:17.751175 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:17.750694    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh" event={"ID":"72282fee-4db2-4178-94cc-2a200921272c","Type":"ContainerStarted","Data":"e17551ddfca8f5afe89f2871510bcbfc28cdb05500d27ed2ff23159e269a2dfd"}
Sep 10 14:02:18.889420 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:18.889390    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"
Sep 10 14:02:19.033326 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:19.033209    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72282fee-4db2-4178-94cc-2a200921272c-maas-api-service-ca\") pod \"72282fee-4db2-4178-94cc-2a200921272c\" (UID: \"72282fee-4db2-4178-94cc-2a200921272c\") "
Sep 10 14:02:19.033648 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:19.033619    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72282fee-4db2-4178-94cc-2a200921272c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72282fee-4db2-4178-94cc-2a200921272c" (UID: "72282fee-4db2-4178-94cc-2a200921272c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:02:19.134857 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:19.134810    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72282fee-4db2-4178-94cc-2a200921272c-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:19.763183 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:19.763148    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"
Sep 10 14:02:19.763361 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:19.763182    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh" event={"ID":"72282fee-4db2-4178-94cc-2a200921272c","Type":"ContainerDied","Data":"e17551ddfca8f5afe89f2871510bcbfc28cdb05500d27ed2ff23159e269a2dfd"}
Sep 10 14:02:19.763361 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:19.763235    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e17551ddfca8f5afe89f2871510bcbfc28cdb05500d27ed2ff23159e269a2dfd"
Sep 10 14:02:27.162968 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.162920    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7"]
Sep 10 14:02:27.171396 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.171358    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fc6ea5-29817480-hfff7"]
Sep 10 14:02:27.212321 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.212284    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"]
Sep 10 14:02:27.212542 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.212518    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt" podUID="703a58d9-b132-41fc-a0be-9e7885f09da0" containerName="payload-processing" containerID="cri-o://cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1" gracePeriod=30
Sep 10 14:02:27.227742 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.227707    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"]
Sep 10 14:02:27.228028 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.227992    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr" podUID="8b703e6e-720e-4674-9ab0-e9bd30160569" containerName="payload-pre-processing" containerID="cri-o://a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d" gracePeriod=30
Sep 10 14:02:27.498341 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.498312    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 14:02:27.501188 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.501163    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 14:02:27.506434 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.506405    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq44b\" (UniqueName: \"kubernetes.io/projected/8b703e6e-720e-4674-9ab0-e9bd30160569-kube-api-access-sq44b\") pod \"8b703e6e-720e-4674-9ab0-e9bd30160569\" (UID: \"8b703e6e-720e-4674-9ab0-e9bd30160569\") "
Sep 10 14:02:27.506589 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.506475    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b703e6e-720e-4674-9ab0-e9bd30160569-plugins-config-volume\") pod \"8b703e6e-720e-4674-9ab0-e9bd30160569\" (UID: \"8b703e6e-720e-4674-9ab0-e9bd30160569\") "
Sep 10 14:02:27.506589 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.506516    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/703a58d9-b132-41fc-a0be-9e7885f09da0-plugins-config-volume\") pod \"703a58d9-b132-41fc-a0be-9e7885f09da0\" (UID: \"703a58d9-b132-41fc-a0be-9e7885f09da0\") "
Sep 10 14:02:27.506589 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.506542    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mx85l\" (UniqueName: \"kubernetes.io/projected/703a58d9-b132-41fc-a0be-9e7885f09da0-kube-api-access-mx85l\") pod \"703a58d9-b132-41fc-a0be-9e7885f09da0\" (UID: \"703a58d9-b132-41fc-a0be-9e7885f09da0\") "
Sep 10 14:02:27.506935 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.506894    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/703a58d9-b132-41fc-a0be-9e7885f09da0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "703a58d9-b132-41fc-a0be-9e7885f09da0" (UID: "703a58d9-b132-41fc-a0be-9e7885f09da0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:02:27.506935 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.506921    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b703e6e-720e-4674-9ab0-e9bd30160569-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8b703e6e-720e-4674-9ab0-e9bd30160569" (UID: "8b703e6e-720e-4674-9ab0-e9bd30160569"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:02:27.508745 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.508710    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b703e6e-720e-4674-9ab0-e9bd30160569-kube-api-access-sq44b" (OuterVolumeSpecName: "kube-api-access-sq44b") pod "8b703e6e-720e-4674-9ab0-e9bd30160569" (UID: "8b703e6e-720e-4674-9ab0-e9bd30160569"). InnerVolumeSpecName "kube-api-access-sq44b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:02:27.508745 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.508732    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/703a58d9-b132-41fc-a0be-9e7885f09da0-kube-api-access-mx85l" (OuterVolumeSpecName: "kube-api-access-mx85l") pod "703a58d9-b132-41fc-a0be-9e7885f09da0" (UID: "703a58d9-b132-41fc-a0be-9e7885f09da0"). InnerVolumeSpecName "kube-api-access-mx85l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:02:27.591908 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.591858    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"]
Sep 10 14:02:27.599082 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.599050    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fc6ea5-t45nh"]
Sep 10 14:02:27.607687 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.607657    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mx85l\" (UniqueName: \"kubernetes.io/projected/703a58d9-b132-41fc-a0be-9e7885f09da0-kube-api-access-mx85l\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:27.607687 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.607683    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq44b\" (UniqueName: \"kubernetes.io/projected/8b703e6e-720e-4674-9ab0-e9bd30160569-kube-api-access-sq44b\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:27.607851 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.607695    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b703e6e-720e-4674-9ab0-e9bd30160569-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:27.607851 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.607703    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/703a58d9-b132-41fc-a0be-9e7885f09da0-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:27.674951 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.674915    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72282fee-4db2-4178-94cc-2a200921272c" path="/var/lib/kubelet/pods/72282fee-4db2-4178-94cc-2a200921272c/volumes"
Sep 10 14:02:27.675229 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.675215    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a20c5f4e-68e2-4d99-8837-a9b25765b962" path="/var/lib/kubelet/pods/a20c5f4e-68e2-4d99-8837-a9b25765b962/volumes"
Sep 10 14:02:27.801830 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.801794    2549 generic.go:356] "Generic (PLEG): container finished" podID="703a58d9-b132-41fc-a0be-9e7885f09da0" containerID="cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1" exitCode=0
Sep 10 14:02:27.802028 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.801843    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"
Sep 10 14:02:27.802028 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.801888    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt" event={"ID":"703a58d9-b132-41fc-a0be-9e7885f09da0","Type":"ContainerDied","Data":"cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1"}
Sep 10 14:02:27.802028 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.801947    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt" event={"ID":"703a58d9-b132-41fc-a0be-9e7885f09da0","Type":"ContainerDied","Data":"7cf2029653b04abf6c02184829170531a7b020c3148b018765277e2ba069a403"}
Sep 10 14:02:27.802028 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.801982    2549 scope.go:117] "RemoveContainer" containerID="cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1"
Sep 10 14:02:27.803400 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.803375    2549 generic.go:356] "Generic (PLEG): container finished" podID="8b703e6e-720e-4674-9ab0-e9bd30160569" containerID="a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d" exitCode=0
Sep 10 14:02:27.803482 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.803413    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"
Sep 10 14:02:27.803482 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.803466    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr" event={"ID":"8b703e6e-720e-4674-9ab0-e9bd30160569","Type":"ContainerDied","Data":"a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d"}
Sep 10 14:02:27.803560 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.803510    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr" event={"ID":"8b703e6e-720e-4674-9ab0-e9bd30160569","Type":"ContainerDied","Data":"080b644b7d206aac8da1278f0d16c49101f6dbaa29d030e22216f12a8b9f56e0"}
Sep 10 14:02:27.811008 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.810988    2549 scope.go:117] "RemoveContainer" containerID="cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1"
Sep 10 14:02:27.811313 ip-10-0-137-53 kubenswrapper[2549]: E0910 14:02:27.811292    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1\": container with ID starting with cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1 not found: ID does not exist" containerID="cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1"
Sep 10 14:02:27.811400 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.811325    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1"} err="failed to get container status \"cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1\": rpc error: code = NotFound desc = could not find container \"cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1\": container with ID starting with cc5c389d87b2b895daa34a16eafa16c25ce1822e798d0b7a3b8d4c1ae6d4bdd1 not found: ID does not exist"
Sep 10 14:02:27.811400 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.811351    2549 scope.go:117] "RemoveContainer" containerID="a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d"
Sep 10 14:02:27.819593 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.819575    2549 scope.go:117] "RemoveContainer" containerID="a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d"
Sep 10 14:02:27.819911 ip-10-0-137-53 kubenswrapper[2549]: E0910 14:02:27.819890    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d\": container with ID starting with a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d not found: ID does not exist" containerID="a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d"
Sep 10 14:02:27.819989 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.819917    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d"} err="failed to get container status \"a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d\": rpc error: code = NotFound desc = could not find container \"a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d\": container with ID starting with a2096efca43ebcc280a8b87aea731baa77e1e803e5ef4f9d490af276edf1548d not found: ID does not exist"
Sep 10 14:02:27.826895 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.826860    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"]
Sep 10 14:02:27.830128 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.830104    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fc6ea5-86cb75996b-cxmvr"]
Sep 10 14:02:27.845003 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.844973    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"]
Sep 10 14:02:27.849183 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:27.849155    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fc6ea5-946fdd767-jm9jt"]
Sep 10 14:02:29.673563 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:29.673525    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="703a58d9-b132-41fc-a0be-9e7885f09da0" path="/var/lib/kubelet/pods/703a58d9-b132-41fc-a0be-9e7885f09da0/volumes"
Sep 10 14:02:29.673969 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:29.673892    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b703e6e-720e-4674-9ab0-e9bd30160569" path="/var/lib/kubelet/pods/8b703e6e-720e-4674-9ab0-e9bd30160569/volumes"
Sep 10 14:02:35.913101 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.913057    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"]
Sep 10 14:02:35.913499 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.913393    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="703a58d9-b132-41fc-a0be-9e7885f09da0" containerName="payload-processing"
Sep 10 14:02:35.913499 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.913405    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="703a58d9-b132-41fc-a0be-9e7885f09da0" containerName="payload-processing"
Sep 10 14:02:35.913499 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.913424    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b703e6e-720e-4674-9ab0-e9bd30160569" containerName="payload-pre-processing"
Sep 10 14:02:35.913499 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.913431    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b703e6e-720e-4674-9ab0-e9bd30160569" containerName="payload-pre-processing"
Sep 10 14:02:35.913499 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.913438    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72282fee-4db2-4178-94cc-2a200921272c" containerName="revoke-keys"
Sep 10 14:02:35.913499 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.913443    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="72282fee-4db2-4178-94cc-2a200921272c" containerName="revoke-keys"
Sep 10 14:02:35.913499 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.913497    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="703a58d9-b132-41fc-a0be-9e7885f09da0" containerName="payload-processing"
Sep 10 14:02:35.913721 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.913507    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b703e6e-720e-4674-9ab0-e9bd30160569" containerName="payload-pre-processing"
Sep 10 14:02:35.913721 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.913513    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="72282fee-4db2-4178-94cc-2a200921272c" containerName="revoke-keys"
Sep 10 14:02:35.917964 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.917943    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"
Sep 10 14:02:35.922324 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.922300    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:02:35.922447 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.922305    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 14:02:35.973994 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.973946    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49336adf-590e-4835-81f7-2cd235ebcbb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k\" (UID: \"49336adf-590e-4835-81f7-2cd235ebcbb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"
Sep 10 14:02:35.983081 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:35.983042    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"]
Sep 10 14:02:36.075264 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:36.075221    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49336adf-590e-4835-81f7-2cd235ebcbb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k\" (UID: \"49336adf-590e-4835-81f7-2cd235ebcbb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"
Sep 10 14:02:36.075903 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:36.075850    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49336adf-590e-4835-81f7-2cd235ebcbb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k\" (UID: \"49336adf-590e-4835-81f7-2cd235ebcbb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"
Sep 10 14:02:36.228307 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:36.228219    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"
Sep 10 14:02:36.502983 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:36.502894    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"]
Sep 10 14:02:36.506164 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:02:36.506117    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49336adf_590e_4835_81f7_2cd235ebcbb6.slice/crio-767dd9a8f7f24acd8a39ac758b9a8636433c798513e25d8cb205959457c4b475 WatchSource:0}: Error finding container 767dd9a8f7f24acd8a39ac758b9a8636433c798513e25d8cb205959457c4b475: Status 404 returned error can't find the container with id 767dd9a8f7f24acd8a39ac758b9a8636433c798513e25d8cb205959457c4b475
Sep 10 14:02:36.854478 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:36.854427    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k" event={"ID":"49336adf-590e-4835-81f7-2cd235ebcbb6","Type":"ContainerStarted","Data":"bc04294cd6bc00085ddc906281a3a7d249406b2033222f5ccfdf96fa4e699e16"}
Sep 10 14:02:36.854701 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:36.854533    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k" event={"ID":"49336adf-590e-4835-81f7-2cd235ebcbb6","Type":"ContainerStarted","Data":"767dd9a8f7f24acd8a39ac758b9a8636433c798513e25d8cb205959457c4b475"}
Sep 10 14:02:36.877079 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:36.877019    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k" podStartSLOduration=1.8770034359999999 podStartE2EDuration="1.877003436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:02:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:02:36.875379986 +0000 UTC m=+1915.781938017" watchObservedRunningTime="2026-09-10 14:02:36.877003436 +0000 UTC m=+1915.783561468"
Sep 10 14:02:37.861604 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:37.861566    2549 generic.go:356] "Generic (PLEG): container finished" podID="49336adf-590e-4835-81f7-2cd235ebcbb6" containerID="bc04294cd6bc00085ddc906281a3a7d249406b2033222f5ccfdf96fa4e699e16" exitCode=0
Sep 10 14:02:37.862017 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:37.861633    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k" event={"ID":"49336adf-590e-4835-81f7-2cd235ebcbb6","Type":"ContainerDied","Data":"bc04294cd6bc00085ddc906281a3a7d249406b2033222f5ccfdf96fa4e699e16"}
Sep 10 14:02:38.989880 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:38.989844    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"
Sep 10 14:02:39.106611 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:39.106571    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49336adf-590e-4835-81f7-2cd235ebcbb6-maas-api-service-ca\") pod \"49336adf-590e-4835-81f7-2cd235ebcbb6\" (UID: \"49336adf-590e-4835-81f7-2cd235ebcbb6\") "
Sep 10 14:02:39.107007 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:39.106987    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49336adf-590e-4835-81f7-2cd235ebcbb6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "49336adf-590e-4835-81f7-2cd235ebcbb6" (UID: "49336adf-590e-4835-81f7-2cd235ebcbb6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:02:39.207597 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:39.207548    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49336adf-590e-4835-81f7-2cd235ebcbb6-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:39.872341 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:39.872309    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"
Sep 10 14:02:39.872532 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:39.872313    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k" event={"ID":"49336adf-590e-4835-81f7-2cd235ebcbb6","Type":"ContainerDied","Data":"767dd9a8f7f24acd8a39ac758b9a8636433c798513e25d8cb205959457c4b475"}
Sep 10 14:02:39.872532 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:39.872447    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="767dd9a8f7f24acd8a39ac758b9a8636433c798513e25d8cb205959457c4b475"
Sep 10 14:02:41.824316 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:41.824283    2549 scope.go:117] "RemoveContainer" containerID="d2f41d9c3ab058209c723c5d9eb93ef13e06790bd04640a353ae8be15bcd29a4"
Sep 10 14:02:41.834797 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:41.834763    2549 scope.go:117] "RemoveContainer" containerID="e550a62fd982e79aa5239fb1984bc1969b7df52738dcb92a42c83f8fcc5b17fd"
Sep 10 14:02:46.614279 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.614239    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"]
Sep 10 14:02:46.614744 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.614495    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56" podUID="5189b16f-1b97-4dbf-939f-41e8a4dc3f6d" containerName="payload-processing" containerID="cri-o://7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d" gracePeriod=30
Sep 10 14:02:46.629442 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.629383    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"]
Sep 10 14:02:46.630230 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.629654    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96" podUID="5859e036-3956-418a-9af2-9df64e00dbf9" containerName="payload-pre-processing" containerID="cri-o://ecd2377bda2aa3fd6cd46fd7a0be96586eb5a8367755514f83ee9c74d1cf8a21" gracePeriod=30
Sep 10 14:02:46.871043 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.870974    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 14:02:46.900153 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.900115    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"]
Sep 10 14:02:46.908078 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.906758    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-579ae1-6km2k"]
Sep 10 14:02:46.914142 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.914104    2549 generic.go:356] "Generic (PLEG): container finished" podID="5189b16f-1b97-4dbf-939f-41e8a4dc3f6d" containerID="7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d" exitCode=0
Sep 10 14:02:46.914285 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.914198    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"
Sep 10 14:02:46.914359 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.914191    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56" event={"ID":"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d","Type":"ContainerDied","Data":"7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d"}
Sep 10 14:02:46.914416 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.914377    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56" event={"ID":"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d","Type":"ContainerDied","Data":"eeb886f626dbc5939ddf432f1bbe162f859f134a3ef39b757804be1c03a7822e"}
Sep 10 14:02:46.914469 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.914418    2549 scope.go:117] "RemoveContainer" containerID="7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d"
Sep 10 14:02:46.915970 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.915944    2549 generic.go:356] "Generic (PLEG): container finished" podID="5859e036-3956-418a-9af2-9df64e00dbf9" containerID="ecd2377bda2aa3fd6cd46fd7a0be96586eb5a8367755514f83ee9c74d1cf8a21" exitCode=0
Sep 10 14:02:46.916057 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.916017    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96" event={"ID":"5859e036-3956-418a-9af2-9df64e00dbf9","Type":"ContainerDied","Data":"ecd2377bda2aa3fd6cd46fd7a0be96586eb5a8367755514f83ee9c74d1cf8a21"}
Sep 10 14:02:46.923584 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.923562    2549 scope.go:117] "RemoveContainer" containerID="7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d"
Sep 10 14:02:46.923857 ip-10-0-137-53 kubenswrapper[2549]: E0910 14:02:46.923838    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d\": container with ID starting with 7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d not found: ID does not exist" containerID="7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d"
Sep 10 14:02:46.923940 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.923905    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d"} err="failed to get container status \"7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d\": rpc error: code = NotFound desc = could not find container \"7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d\": container with ID starting with 7f7e75aab4e5b3f03493431fa3f6781ba18d9ec7b2264fe9cda6ac37b1b0438d not found: ID does not exist"
Sep 10 14:02:46.925316 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.925300    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 14:02:46.975466 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.975433    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnf55\" (UniqueName: \"kubernetes.io/projected/5859e036-3956-418a-9af2-9df64e00dbf9-kube-api-access-lnf55\") pod \"5859e036-3956-418a-9af2-9df64e00dbf9\" (UID: \"5859e036-3956-418a-9af2-9df64e00dbf9\") "
Sep 10 14:02:46.975677 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.975502    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5859e036-3956-418a-9af2-9df64e00dbf9-plugins-config-volume\") pod \"5859e036-3956-418a-9af2-9df64e00dbf9\" (UID: \"5859e036-3956-418a-9af2-9df64e00dbf9\") "
Sep 10 14:02:46.975677 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.975530    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-plugins-config-volume\") pod \"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d\" (UID: \"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d\") "
Sep 10 14:02:46.975677 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.975571    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72bq7\" (UniqueName: \"kubernetes.io/projected/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-kube-api-access-72bq7\") pod \"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d\" (UID: \"5189b16f-1b97-4dbf-939f-41e8a4dc3f6d\") "
Sep 10 14:02:46.975943 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.975908    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5859e036-3956-418a-9af2-9df64e00dbf9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5859e036-3956-418a-9af2-9df64e00dbf9" (UID: "5859e036-3956-418a-9af2-9df64e00dbf9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:02:46.976061 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.975940    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5189b16f-1b97-4dbf-939f-41e8a4dc3f6d" (UID: "5189b16f-1b97-4dbf-939f-41e8a4dc3f6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:02:46.977714 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.977690    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-kube-api-access-72bq7" (OuterVolumeSpecName: "kube-api-access-72bq7") pod "5189b16f-1b97-4dbf-939f-41e8a4dc3f6d" (UID: "5189b16f-1b97-4dbf-939f-41e8a4dc3f6d"). InnerVolumeSpecName "kube-api-access-72bq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:02:46.977802 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:46.977725    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5859e036-3956-418a-9af2-9df64e00dbf9-kube-api-access-lnf55" (OuterVolumeSpecName: "kube-api-access-lnf55") pod "5859e036-3956-418a-9af2-9df64e00dbf9" (UID: "5859e036-3956-418a-9af2-9df64e00dbf9"). InnerVolumeSpecName "kube-api-access-lnf55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:02:47.076961 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.076919    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5859e036-3956-418a-9af2-9df64e00dbf9-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:47.076961 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.076955    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:47.076961 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.076965    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72bq7\" (UniqueName: \"kubernetes.io/projected/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d-kube-api-access-72bq7\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:47.077214 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.076974    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnf55\" (UniqueName: \"kubernetes.io/projected/5859e036-3956-418a-9af2-9df64e00dbf9-kube-api-access-lnf55\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:47.239447 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.239409    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"]
Sep 10 14:02:47.247979 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.247947    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-579ae1-9979545f5-pzb56"]
Sep 10 14:02:47.673857 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.673820    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49336adf-590e-4835-81f7-2cd235ebcbb6" path="/var/lib/kubelet/pods/49336adf-590e-4835-81f7-2cd235ebcbb6/volumes"
Sep 10 14:02:47.674276 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.674200    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5189b16f-1b97-4dbf-939f-41e8a4dc3f6d" path="/var/lib/kubelet/pods/5189b16f-1b97-4dbf-939f-41e8a4dc3f6d/volumes"
Sep 10 14:02:47.923063 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.923030    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96" event={"ID":"5859e036-3956-418a-9af2-9df64e00dbf9","Type":"ContainerDied","Data":"8dbbb1df30c8df2d4c7c1c43309ee40df7dd1202c697109915604a930f0d675e"}
Sep 10 14:02:47.923244 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.923066    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"
Sep 10 14:02:47.923244 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.923094    2549 scope.go:117] "RemoveContainer" containerID="ecd2377bda2aa3fd6cd46fd7a0be96586eb5a8367755514f83ee9c74d1cf8a21"
Sep 10 14:02:47.943460 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.943432    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"]
Sep 10 14:02:47.948016 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:47.947992    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-579ae1-686ff76ff4-s6v96"]
Sep 10 14:02:49.673969 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:49.673938    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5859e036-3956-418a-9af2-9df64e00dbf9" path="/var/lib/kubelet/pods/5859e036-3956-418a-9af2-9df64e00dbf9/volumes"
Sep 10 14:02:55.318160 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.318119    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"]
Sep 10 14:02:55.318542 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.318396    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5859e036-3956-418a-9af2-9df64e00dbf9" containerName="payload-pre-processing"
Sep 10 14:02:55.318542 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.318406    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5859e036-3956-418a-9af2-9df64e00dbf9" containerName="payload-pre-processing"
Sep 10 14:02:55.318542 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.318415    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49336adf-590e-4835-81f7-2cd235ebcbb6" containerName="revoke-keys"
Sep 10 14:02:55.318542 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.318421    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="49336adf-590e-4835-81f7-2cd235ebcbb6" containerName="revoke-keys"
Sep 10 14:02:55.318542 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.318431    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5189b16f-1b97-4dbf-939f-41e8a4dc3f6d" containerName="payload-processing"
Sep 10 14:02:55.318542 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.318437    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5189b16f-1b97-4dbf-939f-41e8a4dc3f6d" containerName="payload-processing"
Sep 10 14:02:55.318542 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.318503    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5859e036-3956-418a-9af2-9df64e00dbf9" containerName="payload-pre-processing"
Sep 10 14:02:55.318542 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.318512    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5189b16f-1b97-4dbf-939f-41e8a4dc3f6d" containerName="payload-processing"
Sep 10 14:02:55.318542 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.318522    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="49336adf-590e-4835-81f7-2cd235ebcbb6" containerName="revoke-keys"
Sep 10 14:02:55.322697 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.322680    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"
Sep 10 14:02:55.325251 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.325227    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h95p6\""
Sep 10 14:02:55.325595 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.325578    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:02:55.331523 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.331498    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"]
Sep 10 14:02:55.442981 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.442942    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f9a9cfc-950d-4dc5-940b-a3d9d96553d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm\" (UID: \"0f9a9cfc-950d-4dc5-940b-a3d9d96553d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"
Sep 10 14:02:55.543817 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.543784    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f9a9cfc-950d-4dc5-940b-a3d9d96553d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm\" (UID: \"0f9a9cfc-950d-4dc5-940b-a3d9d96553d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"
Sep 10 14:02:55.544474 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.544447    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f9a9cfc-950d-4dc5-940b-a3d9d96553d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm\" (UID: \"0f9a9cfc-950d-4dc5-940b-a3d9d96553d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"
Sep 10 14:02:55.633635 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.633549    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"
Sep 10 14:02:55.909089 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.908918    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"]
Sep 10 14:02:55.911813 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:02:55.911787    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f9a9cfc_950d_4dc5_940b_a3d9d96553d0.slice/crio-ec5ec9f43052bb8690cd709ef24a67cb04614abfaebdedae814e2bea9bea0043 WatchSource:0}: Error finding container ec5ec9f43052bb8690cd709ef24a67cb04614abfaebdedae814e2bea9bea0043: Status 404 returned error can't find the container with id ec5ec9f43052bb8690cd709ef24a67cb04614abfaebdedae814e2bea9bea0043
Sep 10 14:02:55.964767 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:55.964736    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm" event={"ID":"0f9a9cfc-950d-4dc5-940b-a3d9d96553d0","Type":"ContainerStarted","Data":"ec5ec9f43052bb8690cd709ef24a67cb04614abfaebdedae814e2bea9bea0043"}
Sep 10 14:02:56.971286 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:56.971252    2549 generic.go:356] "Generic (PLEG): container finished" podID="0f9a9cfc-950d-4dc5-940b-a3d9d96553d0" containerID="970e287a0ada8c62d695d4fdf1d9f2051e70b9b3db67b8af0603ed3f7baccf43" exitCode=0
Sep 10 14:02:56.971707 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:56.971341    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm" event={"ID":"0f9a9cfc-950d-4dc5-940b-a3d9d96553d0","Type":"ContainerDied","Data":"970e287a0ada8c62d695d4fdf1d9f2051e70b9b3db67b8af0603ed3f7baccf43"}
Sep 10 14:02:58.105150 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:58.105105    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"
Sep 10 14:02:58.165304 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:58.165266    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f9a9cfc-950d-4dc5-940b-a3d9d96553d0-maas-api-service-ca\") pod \"0f9a9cfc-950d-4dc5-940b-a3d9d96553d0\" (UID: \"0f9a9cfc-950d-4dc5-940b-a3d9d96553d0\") "
Sep 10 14:02:58.165674 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:58.165645    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f9a9cfc-950d-4dc5-940b-a3d9d96553d0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0f9a9cfc-950d-4dc5-940b-a3d9d96553d0" (UID: "0f9a9cfc-950d-4dc5-940b-a3d9d96553d0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:02:58.266732 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:58.266631    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f9a9cfc-950d-4dc5-940b-a3d9d96553d0-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:02:58.982579 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:58.982533    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm" event={"ID":"0f9a9cfc-950d-4dc5-940b-a3d9d96553d0","Type":"ContainerDied","Data":"ec5ec9f43052bb8690cd709ef24a67cb04614abfaebdedae814e2bea9bea0043"}
Sep 10 14:02:58.982737 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:58.982589    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"
Sep 10 14:02:58.982737 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:02:58.982595    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec5ec9f43052bb8690cd709ef24a67cb04614abfaebdedae814e2bea9bea0043"
Sep 10 14:03:06.334548 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.334512    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"]
Sep 10 14:03:06.334995 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.334755    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259" podUID="4980fb51-e869-4632-93e1-bb80c85660da" containerName="payload-processing" containerID="cri-o://e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec" gracePeriod=30
Sep 10 14:03:06.583969 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.583938    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 14:03:06.620738 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.620651    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"]
Sep 10 14:03:06.624472 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.624445    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-d5c012-fm7lm"]
Sep 10 14:03:06.737222 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.737178    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px69x\" (UniqueName: \"kubernetes.io/projected/4980fb51-e869-4632-93e1-bb80c85660da-kube-api-access-px69x\") pod \"4980fb51-e869-4632-93e1-bb80c85660da\" (UID: \"4980fb51-e869-4632-93e1-bb80c85660da\") "
Sep 10 14:03:06.737405 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.737303    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4980fb51-e869-4632-93e1-bb80c85660da-plugins-config-volume\") pod \"4980fb51-e869-4632-93e1-bb80c85660da\" (UID: \"4980fb51-e869-4632-93e1-bb80c85660da\") "
Sep 10 14:03:06.737712 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.737678    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4980fb51-e869-4632-93e1-bb80c85660da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4980fb51-e869-4632-93e1-bb80c85660da" (UID: "4980fb51-e869-4632-93e1-bb80c85660da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:03:06.739313 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.739289    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4980fb51-e869-4632-93e1-bb80c85660da-kube-api-access-px69x" (OuterVolumeSpecName: "kube-api-access-px69x") pod "4980fb51-e869-4632-93e1-bb80c85660da" (UID: "4980fb51-e869-4632-93e1-bb80c85660da"). InnerVolumeSpecName "kube-api-access-px69x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:03:06.838933 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.838858    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4980fb51-e869-4632-93e1-bb80c85660da-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:03:06.838933 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:06.838928    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px69x\" (UniqueName: \"kubernetes.io/projected/4980fb51-e869-4632-93e1-bb80c85660da-kube-api-access-px69x\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:03:07.020229 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.020191    2549 generic.go:356] "Generic (PLEG): container finished" podID="4980fb51-e869-4632-93e1-bb80c85660da" containerID="e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec" exitCode=0
Sep 10 14:03:07.020229 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.020234    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"
Sep 10 14:03:07.020519 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.020269    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259" event={"ID":"4980fb51-e869-4632-93e1-bb80c85660da","Type":"ContainerDied","Data":"e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec"}
Sep 10 14:03:07.020519 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.020323    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259" event={"ID":"4980fb51-e869-4632-93e1-bb80c85660da","Type":"ContainerDied","Data":"4429e8e40d055097c62d0bed9100cbed7ad1e17e66ec615aca0c47ec7d4ac6ef"}
Sep 10 14:03:07.020519 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.020351    2549 scope.go:117] "RemoveContainer" containerID="e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec"
Sep 10 14:03:07.029224 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.029204    2549 scope.go:117] "RemoveContainer" containerID="e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec"
Sep 10 14:03:07.029508 ip-10-0-137-53 kubenswrapper[2549]: E0910 14:03:07.029486    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec\": container with ID starting with e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec not found: ID does not exist" containerID="e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec"
Sep 10 14:03:07.029575 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.029521    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec"} err="failed to get container status \"e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec\": rpc error: code = NotFound desc = could not find container \"e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec\": container with ID starting with e2587e799f985512895c0b7cb7ada9a2246c23a6f2f86a1ede1a637831ab3dec not found: ID does not exist"
Sep 10 14:03:07.061547 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.061511    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"]
Sep 10 14:03:07.094649 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.094610    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d5c012-569d4d996c-qp259"]
Sep 10 14:03:07.673567 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.673529    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f9a9cfc-950d-4dc5-940b-a3d9d96553d0" path="/var/lib/kubelet/pods/0f9a9cfc-950d-4dc5-940b-a3d9d96553d0/volumes"
Sep 10 14:03:07.673948 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:07.673918    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4980fb51-e869-4632-93e1-bb80c85660da" path="/var/lib/kubelet/pods/4980fb51-e869-4632-93e1-bb80c85660da/volumes"
Sep 10 14:03:41.864988 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:41.864918    2549 scope.go:117] "RemoveContainer" containerID="4eb651e1e03a5b4dc1be74350834ecb7904f13620622ddf7ed660e0de1835f23"
Sep 10 14:03:41.872860 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:03:41.872837    2549 scope.go:117] "RemoveContainer" containerID="4035a1e5c2030cbac68fcad486a27b2039b16526e15e1201bdfda36cae70f292"
Sep 10 14:05:41.641806 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:05:41.641767    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 14:05:41.650517 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:05:41.650495    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b7sv8_96a4f241-7d7a-45e1-b838-e85ce3b4dad1/ovn-acl-logging/0.log"
Sep 10 14:06:41.916926 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:06:41.916894    2549 scope.go:117] "RemoveContainer" containerID="8cfd22b2a38be6d51e7e6278048a43c5d8f7015d2eb5a6bfa0da7445b89f6f89"
Sep 10 14:06:41.924827 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:06:41.924806    2549 scope.go:117] "RemoveContainer" containerID="2fc253d223e59306be6652e31d9b79f2670d4a4ee8f0d2fd2f934c330d6845b2"
Sep 10 14:06:41.931624 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:06:41.931608    2549 scope.go:117] "RemoveContainer" containerID="e1a7cab725c7525b0a592c110b2a5613a9cdeff9326f308cf444035fc356db74"
Sep 10 14:07:38.210363 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.210330    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"]
Sep 10 14:07:38.210914 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.210727    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4980fb51-e869-4632-93e1-bb80c85660da" containerName="payload-processing"
Sep 10 14:07:38.210914 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.210747    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="4980fb51-e869-4632-93e1-bb80c85660da" containerName="payload-processing"
Sep 10 14:07:38.210914 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.210773    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f9a9cfc-950d-4dc5-940b-a3d9d96553d0" containerName="revoke-keys"
Sep 10 14:07:38.210914 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.210781    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f9a9cfc-950d-4dc5-940b-a3d9d96553d0" containerName="revoke-keys"
Sep 10 14:07:38.210914 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.210852    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f9a9cfc-950d-4dc5-940b-a3d9d96553d0" containerName="revoke-keys"
Sep 10 14:07:38.210914 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.210885    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="4980fb51-e869-4632-93e1-bb80c85660da" containerName="payload-processing"
Sep 10 14:07:38.213573 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.213555    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.217083 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.217063    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-898a8cf6-kserve-self-signed-certs\""
Sep 10 14:07:38.232422 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.232401    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"]
Sep 10 14:07:38.253060 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.253029    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-home\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.253168 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.253070    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tmp-dir\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.253168 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.253128    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-model-cache\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.253168 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.253162    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-dshm\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.253277 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.253235    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tls-certs\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.253315 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.253281    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r45g\" (UniqueName: \"kubernetes.io/projected/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-kube-api-access-4r45g\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.354090 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.354051    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tmp-dir\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.354268 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.354100    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-model-cache\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.354268 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.354190    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-dshm\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.354268 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.354211    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tls-certs\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.354268 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.354245    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r45g\" (UniqueName: \"kubernetes.io/projected/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-kube-api-access-4r45g\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.354268 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.354266    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-home\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.354547 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.354518    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tmp-dir\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.354618 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.354542    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-model-cache\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.354618 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.354601    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-home\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.356620 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.356592    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-dshm\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.356838 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.356819    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tls-certs\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.366544 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.366523    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r45g\" (UniqueName: \"kubernetes.io/projected/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-kube-api-access-4r45g\") pod \"e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") " pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.521788 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.521681    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:38.807805 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:38.807702    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"]
Sep 10 14:07:38.812400 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:07:38.812364    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46df2b2a_84b5_4729_b6fe_10c3b20e6f0b.slice/crio-7cb3ccd2932cb64953186ec4d7793a4181fa1bde9849f916c8a758874013038f WatchSource:0}: Error finding container 7cb3ccd2932cb64953186ec4d7793a4181fa1bde9849f916c8a758874013038f: Status 404 returned error can't find the container with id 7cb3ccd2932cb64953186ec4d7793a4181fa1bde9849f916c8a758874013038f
Sep 10 14:07:39.210066 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:39.210025    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8" event={"ID":"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b","Type":"ContainerStarted","Data":"b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90"}
Sep 10 14:07:39.210066 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:39.210064    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8" event={"ID":"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b","Type":"ContainerStarted","Data":"7cb3ccd2932cb64953186ec4d7793a4181fa1bde9849f916c8a758874013038f"}
Sep 10 14:07:39.249432 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:39.249362    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8" podStartSLOduration=1.2493448090000001 podStartE2EDuration="1.249344809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:07:39.248335595 +0000 UTC m=+2218.154893625" watchObservedRunningTime="2026-09-10 14:07:39.249344809 +0000 UTC m=+2218.155902878"
Sep 10 14:07:41.953785 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:41.953749    2549 scope.go:117] "RemoveContainer" containerID="aa4b0198bf4842d479eb9a49278f86b9e2d8f392c79fa3ce45abf372b9f35afb"
Sep 10 14:07:48.522106 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:48.522066    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:48.522573 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:48.522151    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:48.534812 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:48.534786    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:07:49.266600 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:07:49.266572    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:08:07.035775 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.035678    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"]
Sep 10 14:08:07.038930 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.038906    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.064846 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.064814    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-898a8cf6-kserve-self-signed-certs\""
Sep 10 14:08:07.073253 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.073221    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"]
Sep 10 14:08:07.195796 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.195756    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-home\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.195796 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.195797    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af6087e4-5c34-4c96-89b6-e30b97f7995c-tls-certs\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.196051 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.195922    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfj88\" (UniqueName: \"kubernetes.io/projected/af6087e4-5c34-4c96-89b6-e30b97f7995c-kube-api-access-pfj88\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.196051 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.195962    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-tmp-dir\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.196051 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.196025    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-model-cache\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.196051 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.196046    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-dshm\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.297067 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.296972    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af6087e4-5c34-4c96-89b6-e30b97f7995c-tls-certs\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.297067 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.297034    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj88\" (UniqueName: \"kubernetes.io/projected/af6087e4-5c34-4c96-89b6-e30b97f7995c-kube-api-access-pfj88\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.297067 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.297058    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-tmp-dir\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.297341 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.297090    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-model-cache\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.297341 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.297107    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-dshm\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.297341 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.297134    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-home\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.297559 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.297532    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-tmp-dir\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.297629 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.297558    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-model-cache\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.297629 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.297536    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-home\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.299380 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.299363    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-dshm\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.299647 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.299629    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af6087e4-5c34-4c96-89b6-e30b97f7995c-tls-certs\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.321912 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.321857    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfj88\" (UniqueName: \"kubernetes.io/projected/af6087e4-5c34-4c96-89b6-e30b97f7995c-kube-api-access-pfj88\") pod \"e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") " pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.348913 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.348852    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:07.630696 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:07.630669    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"]
Sep 10 14:08:07.633268 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:08:07.633239    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf6087e4_5c34_4c96_89b6_e30b97f7995c.slice/crio-d5a1db81a132a98a57bb49785a0d0c13630b3210097016c10c208452612739cf WatchSource:0}: Error finding container d5a1db81a132a98a57bb49785a0d0c13630b3210097016c10c208452612739cf: Status 404 returned error can't find the container with id d5a1db81a132a98a57bb49785a0d0c13630b3210097016c10c208452612739cf
Sep 10 14:08:08.334652 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:08.334608    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg" event={"ID":"af6087e4-5c34-4c96-89b6-e30b97f7995c","Type":"ContainerStarted","Data":"a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b"}
Sep 10 14:08:08.334652 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:08.334659    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg" event={"ID":"af6087e4-5c34-4c96-89b6-e30b97f7995c","Type":"ContainerStarted","Data":"d5a1db81a132a98a57bb49785a0d0c13630b3210097016c10c208452612739cf"}
Sep 10 14:08:08.380088 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:08.380035    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg" podStartSLOduration=2.380016807 podStartE2EDuration="2.380016807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:08:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:08:08.379475023 +0000 UTC m=+2247.286033054" watchObservedRunningTime="2026-09-10 14:08:08.380016807 +0000 UTC m=+2247.286574884"
Sep 10 14:08:17.349154 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:17.349113    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:17.349154 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:17.349160    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:17.367906 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:17.367849    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:17.390624 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:17.390578    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:29.034130 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:29.034094    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"]
Sep 10 14:08:31.086298 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:31.086258    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"]
Sep 10 14:08:41.974795 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:41.974756    2549 scope.go:117] "RemoveContainer" containerID="bc04294cd6bc00085ddc906281a3a7d249406b2033222f5ccfdf96fa4e699e16"
Sep 10 14:08:41.982681 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:41.982654    2549 scope.go:117] "RemoveContainer" containerID="714c3b9cbeb932de7ad22f5a97bef3d2d1fc3b31673cc9934e59625610d45f58"
Sep 10 14:08:44.053555 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.053519    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8" podUID="46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" containerName="main" containerID="cri-o://b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90" gracePeriod=45
Sep 10 14:08:44.300445 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.300418    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:08:44.399701 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.399604    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tls-certs\") pod \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") "
Sep 10 14:08:44.399701 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.399670    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r45g\" (UniqueName: \"kubernetes.io/projected/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-kube-api-access-4r45g\") pod \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") "
Sep 10 14:08:44.399942 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.399711    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-home\") pod \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") "
Sep 10 14:08:44.399942 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.399741    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tmp-dir\") pod \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") "
Sep 10 14:08:44.399942 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.399766    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-model-cache\") pod \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") "
Sep 10 14:08:44.399942 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.399795    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-dshm\") pod \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\" (UID: \"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b\") "
Sep 10 14:08:44.400165 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.399995    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-home" (OuterVolumeSpecName: "home") pod "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" (UID: "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:08:44.400165 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.400048    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" (UID: "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:08:44.400165 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.400080    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-model-cache" (OuterVolumeSpecName: "model-cache") pod "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" (UID: "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:08:44.401958 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.401921    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-dshm" (OuterVolumeSpecName: "dshm") pod "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" (UID: "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:08:44.401958 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.401936    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" (UID: "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:08:44.402125 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.402022    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-kube-api-access-4r45g" (OuterVolumeSpecName: "kube-api-access-4r45g") pod "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" (UID: "46df2b2a-84b5-4729-b6fe-10c3b20e6f0b"). InnerVolumeSpecName "kube-api-access-4r45g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:08:44.496601 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.496563    2549 generic.go:356] "Generic (PLEG): container finished" podID="46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" containerID="b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90" exitCode=0
Sep 10 14:08:44.496601 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.496603    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8" event={"ID":"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b","Type":"ContainerDied","Data":"b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90"}
Sep 10 14:08:44.496828 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.496612    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"
Sep 10 14:08:44.496828 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.496632    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8" event={"ID":"46df2b2a-84b5-4729-b6fe-10c3b20e6f0b","Type":"ContainerDied","Data":"7cb3ccd2932cb64953186ec4d7793a4181fa1bde9849f916c8a758874013038f"}
Sep 10 14:08:44.496828 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.496649    2549 scope.go:117] "RemoveContainer" containerID="b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90"
Sep 10 14:08:44.501278 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.501249    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:44.501398 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.501280    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:44.501398 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.501295    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:44.501398 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.501307    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:44.501398 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.501321    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:44.501398 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.501334    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r45g\" (UniqueName: \"kubernetes.io/projected/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b-kube-api-access-4r45g\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:44.505563 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.505539    2549 scope.go:117] "RemoveContainer" containerID="b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90"
Sep 10 14:08:44.505895 ip-10-0-137-53 kubenswrapper[2549]: E0910 14:08:44.505876    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90\": container with ID starting with b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90 not found: ID does not exist" containerID="b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90"
Sep 10 14:08:44.505957 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.505903    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90"} err="failed to get container status \"b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90\": rpc error: code = NotFound desc = could not find container \"b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90\": container with ID starting with b5da4c6cc45cda000753bd7b8716f4b3e3a63b64dfedcc0f23fa348b5b134a90 not found: ID does not exist"
Sep 10 14:08:44.524513 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.524478    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"]
Sep 10 14:08:44.534804 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:44.534766    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-898a8cf6-kserve-8497f8dfb7-tr7w8"]
Sep 10 14:08:45.673740 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:45.673689    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" path="/var/lib/kubelet/pods/46df2b2a-84b5-4729-b6fe-10c3b20e6f0b/volumes"
Sep 10 14:08:46.105479 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.105392    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg" podUID="af6087e4-5c34-4c96-89b6-e30b97f7995c" containerName="main" containerID="cri-o://a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b" gracePeriod=45
Sep 10 14:08:46.361248 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.361178    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:46.419809 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.419769    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-tmp-dir\") pod \"af6087e4-5c34-4c96-89b6-e30b97f7995c\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") "
Sep 10 14:08:46.419809 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.419815    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-home\") pod \"af6087e4-5c34-4c96-89b6-e30b97f7995c\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") "
Sep 10 14:08:46.420089 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.419833    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfj88\" (UniqueName: \"kubernetes.io/projected/af6087e4-5c34-4c96-89b6-e30b97f7995c-kube-api-access-pfj88\") pod \"af6087e4-5c34-4c96-89b6-e30b97f7995c\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") "
Sep 10 14:08:46.420089 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.419913    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-model-cache\") pod \"af6087e4-5c34-4c96-89b6-e30b97f7995c\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") "
Sep 10 14:08:46.420089 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.419940    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-dshm\") pod \"af6087e4-5c34-4c96-89b6-e30b97f7995c\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") "
Sep 10 14:08:46.420089 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.419964    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af6087e4-5c34-4c96-89b6-e30b97f7995c-tls-certs\") pod \"af6087e4-5c34-4c96-89b6-e30b97f7995c\" (UID: \"af6087e4-5c34-4c96-89b6-e30b97f7995c\") "
Sep 10 14:08:46.420289 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.420125    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-home" (OuterVolumeSpecName: "home") pod "af6087e4-5c34-4c96-89b6-e30b97f7995c" (UID: "af6087e4-5c34-4c96-89b6-e30b97f7995c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:08:46.420289 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.420131    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "af6087e4-5c34-4c96-89b6-e30b97f7995c" (UID: "af6087e4-5c34-4c96-89b6-e30b97f7995c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:08:46.420289 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.420220    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-model-cache" (OuterVolumeSpecName: "model-cache") pod "af6087e4-5c34-4c96-89b6-e30b97f7995c" (UID: "af6087e4-5c34-4c96-89b6-e30b97f7995c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:08:46.422137 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.422104    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af6087e4-5c34-4c96-89b6-e30b97f7995c-kube-api-access-pfj88" (OuterVolumeSpecName: "kube-api-access-pfj88") pod "af6087e4-5c34-4c96-89b6-e30b97f7995c" (UID: "af6087e4-5c34-4c96-89b6-e30b97f7995c"). InnerVolumeSpecName "kube-api-access-pfj88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:08:46.422137 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.422131    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af6087e4-5c34-4c96-89b6-e30b97f7995c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af6087e4-5c34-4c96-89b6-e30b97f7995c" (UID: "af6087e4-5c34-4c96-89b6-e30b97f7995c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:08:46.422311 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.422161    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-dshm" (OuterVolumeSpecName: "dshm") pod "af6087e4-5c34-4c96-89b6-e30b97f7995c" (UID: "af6087e4-5c34-4c96-89b6-e30b97f7995c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:08:46.508375 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.508334    2549 generic.go:356] "Generic (PLEG): container finished" podID="af6087e4-5c34-4c96-89b6-e30b97f7995c" containerID="a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b" exitCode=0
Sep 10 14:08:46.508375 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.508375    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"
Sep 10 14:08:46.508600 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.508420    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg" event={"ID":"af6087e4-5c34-4c96-89b6-e30b97f7995c","Type":"ContainerDied","Data":"a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b"}
Sep 10 14:08:46.508600 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.508460    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg" event={"ID":"af6087e4-5c34-4c96-89b6-e30b97f7995c","Type":"ContainerDied","Data":"d5a1db81a132a98a57bb49785a0d0c13630b3210097016c10c208452612739cf"}
Sep 10 14:08:46.508600 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.508480    2549 scope.go:117] "RemoveContainer" containerID="a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b"
Sep 10 14:08:46.517192 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.517170    2549 scope.go:117] "RemoveContainer" containerID="a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b"
Sep 10 14:08:46.517491 ip-10-0-137-53 kubenswrapper[2549]: E0910 14:08:46.517473    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b\": container with ID starting with a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b not found: ID does not exist" containerID="a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b"
Sep 10 14:08:46.517531 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.517501    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b"} err="failed to get container status \"a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b\": rpc error: code = NotFound desc = could not find container \"a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b\": container with ID starting with a864c370507a26591c68e0ac2ef84f371bc423e879f9aad7bb97ca1641bab89b not found: ID does not exist"
Sep 10 14:08:46.520592 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.520576    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:46.520666 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.520594    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:46.520666 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.520604    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af6087e4-5c34-4c96-89b6-e30b97f7995c-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:46.520666 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.520612    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:46.520666 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.520619    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af6087e4-5c34-4c96-89b6-e30b97f7995c-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:46.520666 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.520627    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfj88\" (UniqueName: \"kubernetes.io/projected/af6087e4-5c34-4c96-89b6-e30b97f7995c-kube-api-access-pfj88\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 10 14:08:46.543603 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.543569    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"]
Sep 10 14:08:46.550988 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:46.550958    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-898a8cf6-kserve-6646fd479-7jwlg"]
Sep 10 14:08:47.673982 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:08:47.673937    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af6087e4-5c34-4c96-89b6-e30b97f7995c" path="/var/lib/kubelet/pods/af6087e4-5c34-4c96-89b6-e30b97f7995c/volumes"
Sep 10 14:09:23.114782 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:23.114749    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7477b75cb6-lnfdk_318acfbd-8a1d-487a-9548-9ca5808e446f/authorino/0.log"
Sep 10 14:09:29.360920 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:29.360884    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94d598c87-fdn5q_ae74f9c9-a360-443b-b9c1-07457225c146/manager/0.log"
Sep 10 14:09:29.381151 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:29.381123    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94d598c87-fdn5q_ae74f9c9-a360-443b-b9c1-07457225c146/copy-manifests/0.log"
Sep 10 14:09:30.195779 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:30.195751    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817465-57l6v_d00cadbf-15b9-4d92-89d6-b96d7137fda1/cleanup/0.log"
Sep 10 14:09:30.340461 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:30.340426    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817480-srh9c_c5c8f6c0-3e14-449e-9015-7e889c28c430/cleanup/0.log"
Sep 10 14:09:31.279017 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:31.278986    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7477b75cb6-lnfdk_318acfbd-8a1d-487a-9548-9ca5808e446f/authorino/0.log"
Sep 10 14:09:31.643540 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:31.643466    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-9wg8q_5d810f3c-dfc9-431d-9d25-963f8eedb69f/kuadrant-console-plugin/0.log"
Sep 10 14:09:32.601072 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:32.601043    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66cb58b9b9-2kghf_735169af-6ec4-421c-a26a-22c3532b15ff/kube-auth-proxy/0.log"
Sep 10 14:09:32.981116 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:32.981087    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rvwwf_4dc4baee-8cc6-4af3-bf61-dd80ed85b541/payload-pre-processing/0.log"
Sep 10 14:09:33.845586 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:33.845500    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-sjtzr_358c4045-aa79-4c46-9c39-2dc5c1935411/main/0.log"
Sep 10 14:09:33.964284 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:33.964255    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-4q6rk_a0602337-9751-4c54-9255-9fd9b88973e1/main/0.log"
Sep 10 14:09:34.356029 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:34.355996    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-lj2h4_9ea64d04-56fc-413f-995a-511db7c72b67/main/0.log"
Sep 10 14:09:39.984896 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:39.984847    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mbn6s/must-gather-pzkrl"]
Sep 10 14:09:39.985308 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:39.985136    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af6087e4-5c34-4c96-89b6-e30b97f7995c" containerName="main"
Sep 10 14:09:39.985308 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:39.985148    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="af6087e4-5c34-4c96-89b6-e30b97f7995c" containerName="main"
Sep 10 14:09:39.985308 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:39.985161    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" containerName="main"
Sep 10 14:09:39.985308 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:39.985166    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" containerName="main"
Sep 10 14:09:39.985308 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:39.985216    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="af6087e4-5c34-4c96-89b6-e30b97f7995c" containerName="main"
Sep 10 14:09:39.985308 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:39.985226    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="46df2b2a-84b5-4729-b6fe-10c3b20e6f0b" containerName="main"
Sep 10 14:09:39.987230 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:39.987211    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mbn6s/must-gather-pzkrl"
Sep 10 14:09:39.989958 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:39.989925    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mbn6s\"/\"openshift-service-ca.crt\""
Sep 10 14:09:39.989958 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:39.989947    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mbn6s\"/\"kube-root-ca.crt\""
Sep 10 14:09:40.006667 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.006623    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mbn6s/must-gather-pzkrl"]
Sep 10 14:09:40.153939 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.153897    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b12c7df4-4cf8-43d7-918a-d4b0e8f22d43-must-gather-output\") pod \"must-gather-pzkrl\" (UID: \"b12c7df4-4cf8-43d7-918a-d4b0e8f22d43\") " pod="openshift-must-gather-mbn6s/must-gather-pzkrl"
Sep 10 14:09:40.154128 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.153956    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrt5g\" (UniqueName: \"kubernetes.io/projected/b12c7df4-4cf8-43d7-918a-d4b0e8f22d43-kube-api-access-jrt5g\") pod \"must-gather-pzkrl\" (UID: \"b12c7df4-4cf8-43d7-918a-d4b0e8f22d43\") " pod="openshift-must-gather-mbn6s/must-gather-pzkrl"
Sep 10 14:09:40.254832 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.254733    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b12c7df4-4cf8-43d7-918a-d4b0e8f22d43-must-gather-output\") pod \"must-gather-pzkrl\" (UID: \"b12c7df4-4cf8-43d7-918a-d4b0e8f22d43\") " pod="openshift-must-gather-mbn6s/must-gather-pzkrl"
Sep 10 14:09:40.254832 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.254795    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrt5g\" (UniqueName: \"kubernetes.io/projected/b12c7df4-4cf8-43d7-918a-d4b0e8f22d43-kube-api-access-jrt5g\") pod \"must-gather-pzkrl\" (UID: \"b12c7df4-4cf8-43d7-918a-d4b0e8f22d43\") " pod="openshift-must-gather-mbn6s/must-gather-pzkrl"
Sep 10 14:09:40.255114 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.255093    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b12c7df4-4cf8-43d7-918a-d4b0e8f22d43-must-gather-output\") pod \"must-gather-pzkrl\" (UID: \"b12c7df4-4cf8-43d7-918a-d4b0e8f22d43\") " pod="openshift-must-gather-mbn6s/must-gather-pzkrl"
Sep 10 14:09:40.265019 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.264982    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrt5g\" (UniqueName: \"kubernetes.io/projected/b12c7df4-4cf8-43d7-918a-d4b0e8f22d43-kube-api-access-jrt5g\") pod \"must-gather-pzkrl\" (UID: \"b12c7df4-4cf8-43d7-918a-d4b0e8f22d43\") " pod="openshift-must-gather-mbn6s/must-gather-pzkrl"
Sep 10 14:09:40.296625 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.296592    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mbn6s/must-gather-pzkrl"
Sep 10 14:09:40.573703 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.573622    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mbn6s/must-gather-pzkrl"]
Sep 10 14:09:40.577234 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:09:40.577207    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb12c7df4_4cf8_43d7_918a_d4b0e8f22d43.slice/crio-6fbd6876e5d651b2863333f9329d1994ba7c35a6906a7bfdd88cf76ea617f644 WatchSource:0}: Error finding container 6fbd6876e5d651b2863333f9329d1994ba7c35a6906a7bfdd88cf76ea617f644: Status 404 returned error can't find the container with id 6fbd6876e5d651b2863333f9329d1994ba7c35a6906a7bfdd88cf76ea617f644
Sep 10 14:09:40.579255 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.579237    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:09:40.738701 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:40.738663    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mbn6s/must-gather-pzkrl" event={"ID":"b12c7df4-4cf8-43d7-918a-d4b0e8f22d43","Type":"ContainerStarted","Data":"6fbd6876e5d651b2863333f9329d1994ba7c35a6906a7bfdd88cf76ea617f644"}
Sep 10 14:09:41.746447 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:41.746414    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mbn6s/must-gather-pzkrl" event={"ID":"b12c7df4-4cf8-43d7-918a-d4b0e8f22d43","Type":"ContainerStarted","Data":"534fb6bed139a53a57c3c351aba095cf89e609def42d1beb5642f1d529a265b5"}
Sep 10 14:09:42.005849 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:42.005814    2549 scope.go:117] "RemoveContainer" containerID="970e287a0ada8c62d695d4fdf1d9f2051e70b9b3db67b8af0603ed3f7baccf43"
Sep 10 14:09:42.755290 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:42.755248    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mbn6s/must-gather-pzkrl" event={"ID":"b12c7df4-4cf8-43d7-918a-d4b0e8f22d43","Type":"ContainerStarted","Data":"341789c1da24d74505c5c535bcc6b4f1e045364080011cf57c3f5d1c2a4c89fc"}
Sep 10 14:09:42.774495 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:42.774416    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mbn6s/must-gather-pzkrl" podStartSLOduration=2.817822418 podStartE2EDuration="3.774398885s" totalImagesPullingTime="956.576467ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:09:42.772214801 +0000 UTC m=+2341.678772836" watchObservedRunningTime="2026-09-10 14:09:42.774398885 +0000 UTC m=+2341.680956916"
Sep 10 14:09:43.377323 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:43.377289    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7jhj2_fa5af7a5-42e5-4712-8eb8-cb5385bc6d6e/global-pull-secret-syncer/0.log"
Sep 10 14:09:43.524748 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:43.524719    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tdx8g_69d18837-83dd-4960-a093-eabc75e22ddb/konnectivity-agent/0.log"
Sep 10 14:09:43.737535 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:43.737508    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-53.ec2.internal_651cc6668ff972ae51b87e7dde9a4f79/haproxy/0.log"
Sep 10 14:09:48.571884 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:48.571382    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7477b75cb6-lnfdk_318acfbd-8a1d-487a-9548-9ca5808e446f/authorino/0.log"
Sep 10 14:09:49.015236 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:49.015206    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-9wg8q_5d810f3c-dfc9-431d-9d25-963f8eedb69f/kuadrant-console-plugin/0.log"
Sep 10 14:09:51.616371 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:51.616329    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h5v99_2e581b5d-2326-4c11-8b0b-2d79d91a28c0/node-exporter/0.log"
Sep 10 14:09:51.645662 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:51.645631    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h5v99_2e581b5d-2326-4c11-8b0b-2d79d91a28c0/kube-rbac-proxy/0.log"
Sep 10 14:09:51.674648 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:51.674599    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h5v99_2e581b5d-2326-4c11-8b0b-2d79d91a28c0/init-textfile/0.log"
Sep 10 14:09:54.663556 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.663514    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"]
Sep 10 14:09:54.667618 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.667584    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.670119 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.670087    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mbn6s\"/\"default-dockercfg-2lwpv\""
Sep 10 14:09:54.681448 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.681412    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"]
Sep 10 14:09:54.703888 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.703385    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-proc\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.703888 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.703456    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-lib-modules\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.703888 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.703491    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-sys\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.703888 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.703515    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-podres\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.703888 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.703605    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvttz\" (UniqueName: \"kubernetes.io/projected/26b3b9cf-dd38-42ba-9768-bac94215e6b6-kube-api-access-zvttz\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.804343 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.804300    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-lib-modules\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.804617 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.804597    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-sys\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.804744 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.804728    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-podres\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.804967 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.804948    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvttz\" (UniqueName: \"kubernetes.io/projected/26b3b9cf-dd38-42ba-9768-bac94215e6b6-kube-api-access-zvttz\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.805131 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.805113    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-proc\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.805327 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.805308    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-proc\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.805524 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.805506    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-sys\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.805640 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.805583    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-podres\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.805726 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.804544    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26b3b9cf-dd38-42ba-9768-bac94215e6b6-lib-modules\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.819942 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.819851    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvttz\" (UniqueName: \"kubernetes.io/projected/26b3b9cf-dd38-42ba-9768-bac94215e6b6-kube-api-access-zvttz\") pod \"perf-node-gather-daemonset-ngmns\" (UID: \"26b3b9cf-dd38-42ba-9768-bac94215e6b6\") " pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:54.980186 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:54.980062    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:55.272902 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:55.272795    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"]
Sep 10 14:09:55.276008 ip-10-0-137-53 kubenswrapper[2549]: W0910 14:09:55.275973    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod26b3b9cf_dd38_42ba_9768_bac94215e6b6.slice/crio-e05d2ea176be6c32ae9805f9673f92a503e506a6b407995d38c862945dfb50b9 WatchSource:0}: Error finding container e05d2ea176be6c32ae9805f9673f92a503e506a6b407995d38c862945dfb50b9: Status 404 returned error can't find the container with id e05d2ea176be6c32ae9805f9673f92a503e506a6b407995d38c862945dfb50b9
Sep 10 14:09:55.835414 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:55.835380    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns" event={"ID":"26b3b9cf-dd38-42ba-9768-bac94215e6b6","Type":"ContainerStarted","Data":"cd781adc36cf5b03b1b1b6b916a0f6a5fc6686eed02bdab11ad0a176ab7b772d"}
Sep 10 14:09:55.835812 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:55.835435    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns" event={"ID":"26b3b9cf-dd38-42ba-9768-bac94215e6b6","Type":"ContainerStarted","Data":"e05d2ea176be6c32ae9805f9673f92a503e506a6b407995d38c862945dfb50b9"}
Sep 10 14:09:55.835812 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:55.835513    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:09:55.856082 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:55.856029    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns" podStartSLOduration=1.8560139530000002 podStartE2EDuration="1.856013953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:09:55.855447114 +0000 UTC m=+2354.762005156" watchObservedRunningTime="2026-09-10 14:09:55.856013953 +0000 UTC m=+2354.762571984"
Sep 10 14:09:56.989688 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:56.989652    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dbz76_94abed11-8633-43cc-98a0-19223098fa40/dns/0.log"
Sep 10 14:09:57.037425 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:57.037398    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dbz76_94abed11-8633-43cc-98a0-19223098fa40/kube-rbac-proxy/0.log"
Sep 10 14:09:57.229376 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:57.229349    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pw9wz_dc277d0a-0d17-499c-96e8-50582c9a2655/dns-node-resolver/0.log"
Sep 10 14:09:57.871890 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:57.871843    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c947ff77-zhw2x_3d578d4a-e2a6-46c5-b1eb-6384a3509f2b/registry/0.log"
Sep 10 14:09:57.895976 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:57.895940    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vkmn_2e530ccd-d80f-4199-8b0d-5de767763253/node-ca/0.log"
Sep 10 14:09:59.464057 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:59.464024    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66cb58b9b9-2kghf_735169af-6ec4-421c-a26a-22c3532b15ff/kube-auth-proxy/0.log"
Sep 10 14:09:59.581722 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:09:59.581683    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rvwwf_4dc4baee-8cc6-4af3-bf61-dd80ed85b541/payload-pre-processing/0.log"
Sep 10 14:10:00.618331 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:00.618301    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bhpvl_c1f29aa0-318e-4f86-8500-4e3b68b848f7/serve-healthcheck-canary/0.log"
Sep 10 14:10:01.365749 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:01.365719    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5wjmf_b42d2757-0d53-4232-8a6a-2039f0b93e0f/kube-rbac-proxy/0.log"
Sep 10 14:10:01.405387 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:01.405332    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5wjmf_b42d2757-0d53-4232-8a6a-2039f0b93e0f/exporter/0.log"
Sep 10 14:10:01.432353 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:01.432322    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5wjmf_b42d2757-0d53-4232-8a6a-2039f0b93e0f/extractor/0.log"
Sep 10 14:10:01.850968 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:01.850939    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mbn6s/perf-node-gather-daemonset-ngmns"
Sep 10 14:10:04.258909 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:04.258793    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66cb58b9b9-2kghf_735169af-6ec4-421c-a26a-22c3532b15ff/kube-auth-proxy/0.log"
Sep 10 14:10:04.403499 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:04.403462    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rvwwf_4dc4baee-8cc6-4af3-bf61-dd80ed85b541/payload-pre-processing/0.log"
Sep 10 14:10:05.035687 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:05.035657    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94d598c87-fdn5q_ae74f9c9-a360-443b-b9c1-07457225c146/manager/0.log"
Sep 10 14:10:05.068426 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:05.068395    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94d598c87-fdn5q_ae74f9c9-a360-443b-b9c1-07457225c146/copy-manifests/0.log"
Sep 10 14:10:06.453078 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:06.453049    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66cb58b9b9-2kghf_735169af-6ec4-421c-a26a-22c3532b15ff/kube-auth-proxy/0.log"
Sep 10 14:10:06.537292 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:06.537253    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rvwwf_4dc4baee-8cc6-4af3-bf61-dd80ed85b541/payload-pre-processing/0.log"
Sep 10 14:10:06.729859 ip-10-0-137-53 kubenswrapper[2549]: I0910 14:10:06.729773    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59689cfc58-fvqwx_66b89271-8918-44c5-b25b-630780289f26/manager/0.log"