Sep 18 18:42:17.527870 ip-10-0-136-52 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 18:42:17.527882 ip-10-0-136-52 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 18:42:17.527893 ip-10-0-136-52 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 18:42:17.528266 ip-10-0-136-52 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 18:42:27.571386 ip-10-0-136-52 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 18:42:27.571403 ip-10-0-136-52 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a922ea38578748a0be8934f967785cea --
Sep 18 18:44:23.668066 ip-10-0-136-52 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 18:44:24.108864 ip-10-0-136-52 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:44:24.108864 ip-10-0-136-52 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 18:44:24.108864 ip-10-0-136-52 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:44:24.108864 ip-10-0-136-52 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 18:44:24.108864 ip-10-0-136-52 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:44:24.109659 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.108955    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 18:44:24.113315 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113286    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:24.113315 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113313    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:24.113315 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113318    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113322    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113325    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113328    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113331    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113334    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113337    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113339    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113344    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113347    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113350    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113353    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113356    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113360    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113362    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113365    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113368    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113370    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113373    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113375    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:24.113421 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113378    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113381    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113383    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113386    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113388    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113391    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113393    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113396    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113399    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113402    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113405    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113407    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113410    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113413    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113415    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113418    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113420    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113423    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113425    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113428    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:24.113899 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113431    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113433    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113436    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113439    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113442    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113445    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113447    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113449    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113452    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113454    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113457    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113459    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113461    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113464    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113467    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113469    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113472    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113474    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113477    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113479    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:24.114380 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113482    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113485    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113488    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113490    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113492    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113495    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113498    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113500    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113503    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113505    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113508    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113510    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113513    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113515    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113517    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113520    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113523    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113525    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113528    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113531    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:24.114874 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113533    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113535    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113538    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113541    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113543    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113546    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113548    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113551    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113553    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113556    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113558    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113560    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113566    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113569    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113573    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113576    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.113578    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114305    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114312    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:24.115344 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114314    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114317    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114319    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114322    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114324    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114327    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114330    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114332    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114334    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114338    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114340    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114343    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114346    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114349    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114352    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114355    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114358    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114360    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114362    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:24.115845 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114366    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114370    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114373    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114376    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114379    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114381    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114385    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114389    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114393    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114396    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114400    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114403    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114406    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114409    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114411    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114413    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114416    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114418    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114421    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:24.116322 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114423    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114426    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114428    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114430    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114433    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114436    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114438    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114441    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114443    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114446    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114449    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114452    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114454    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114456    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114459    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114461    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114464    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114466    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114469    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114471    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:24.116802 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114474    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114477    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114479    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114482    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114484    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114487    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114489    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114491    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114494    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114496    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114499    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114501    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114503    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114506    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114508    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114511    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114513    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114516    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114519    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114521    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:24.117287 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114523    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114526    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114529    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114531    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114533    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114536    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114538    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114540    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114543    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114545    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114548    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114551    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114553    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114556    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114559    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114561    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114564    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114566    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.114568    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115010    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:24.117779 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115016    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115019    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115022    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115025    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115028    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115030    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115033    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115035    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115038    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115041    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115044    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115047    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115049    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115052    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115056    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115059    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115063    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115065    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115068    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:24.118259 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115070    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115072    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115075    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115078    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115080    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115083    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115085    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115089    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115091    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115093    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115096    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115098    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115101    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115104    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115107    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115110    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115113    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115115    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115118    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115120    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:24.118722 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115122    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115127    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115129    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115132    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115134    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115137    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115140    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115142    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115144    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115147    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115149    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115152    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115154    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115157    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115159    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115161    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115164    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115166    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115168    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115171    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:24.119221 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115174    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115176    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115178    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115181    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115183    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115186    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115188    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115191    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115193    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115195    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115198    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115200    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115202    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115208    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115211    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115214    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115217    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115221    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115223    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115226    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:24.119741 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115228    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115231    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115233    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115236    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115238    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115241    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115243    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115246    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115248    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115251    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115254    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115256    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115258    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115261    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115263    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115265    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115268    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115270    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.115272    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115351    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 18:44:24.120209 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115412    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115418    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115423    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115427    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115431    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115437    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115442    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115446    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115449    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115454    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115457    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115460    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115463    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115466    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115469    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115472    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115475    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115480    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115483    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115486    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115489    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115493    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115496    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 18:44:24.120791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115500    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115503    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115506    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115509    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115512    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115516    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115519    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115521    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115525    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115528    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115531    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115534    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115537    2581 flags.go:64] FLAG: --enable-server="true"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115540    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115546    2581 flags.go:64] FLAG: --event-burst="100"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115549    2581 flags.go:64] FLAG: --event-qps="50"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115552    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115555    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115558    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115562    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115565    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115568    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115571    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115574    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115576    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 18:44:24.121330 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115579    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115582    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115585    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115588    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115591    2581 flags.go:64] FLAG: --feature-gates=""
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115594    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115597    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115600    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115603    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115606    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115609    2581 flags.go:64] FLAG: --help="false"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115614    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115617    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115620    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115623    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115626    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115629    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115632    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115635    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115638    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115640    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115644    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115648    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115650    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 18 18:44:24.121973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115653    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115656    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115659    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115662    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115665    2581 flags.go:64] FLAG: --lock-file=""
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115668    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115671    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115674    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115680    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115683    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115685    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115688    2581 flags.go:64] FLAG: --logging-format="text"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115691    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115694    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115697    2581 flags.go:64] FLAG: --manifest-url=""
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115715    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115719    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115722    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115726    2581 flags.go:64] FLAG: --max-pods="110"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115733    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115736    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115739    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115742    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115745    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115748    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 18:44:24.122537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115751    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115759    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115762    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115765    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115769    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115772    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115778    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115782    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115785    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115788    2581 flags.go:64] FLAG: --port="10250"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115791    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115794    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ec62844c50153a9e"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115798    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115800    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115803    2581 flags.go:64] FLAG: --register-node="true"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115806    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115810    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115813    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115816    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115819    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115823    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115826    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115829    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115832    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115835    2581 flags.go:64] FLAG: --runonce="false"
Sep 18 18:44:24.123198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115838    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115842    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115845    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115848    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115851    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115855    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115858    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115861    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115864    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115867    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115870    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115873    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115878    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115881    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115883    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115889    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115892    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115895    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115900    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115903    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115906    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115909    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115912    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115916    2581 flags.go:64] FLAG: --v="2"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115920    2581 flags.go:64] FLAG: --version="false"
Sep 18 18:44:24.123859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115924    2581 flags.go:64] FLAG: --vmodule=""
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115928    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.115931    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116050    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116055    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116058    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116060    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116064    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116068    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116071    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116074    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116077    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116080    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116083    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116085    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116088    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116090    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116093    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116095    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116099    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:24.124480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116102    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116105    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116107    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116110    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116113    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116115    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116118    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116121    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116124    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116127    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116130    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116133    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116136    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116138    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116141    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116143    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116146    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116149    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116151    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:24.124993 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116154    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116158    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116160    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116163    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116166    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116168    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116171    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116173    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116176    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116179    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116181    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116183    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116187    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116190    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116192    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116195    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116198    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116200    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116203    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116206    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:24.125443 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116208    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116211    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116214    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116217    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116219    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116222    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116224    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116227    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116229    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116232    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116235    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116239    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116241    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116245    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116248    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116251    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116254    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116256    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116259    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116261    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:24.125944 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116264    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116266    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116269    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116271    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116275    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116278    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116282    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116285    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116288    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116291    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116294    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116297    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116300    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116315    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116318    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116321    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116324    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116327    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116330    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116353    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:24.126414 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116357    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116360    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.116363    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.117122    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.125269    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.125289    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.125305    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.125311    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125369    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125374    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125377    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125380    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125383    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125386    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125389    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:24.126896 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125392    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125394    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125397    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125399    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125402    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125404    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125406    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125409    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125412    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125416    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125420    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125423    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125426    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125429    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125431    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125434    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125437    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125439    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125443    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:24.127300 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125446    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125449    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125452    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125457    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125460    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125463    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125467    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125470    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125473    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125476    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125478    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125481    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125483    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125485    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125488    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125490    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125493    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125495    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125498    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125500    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:24.127810 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125503    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125505    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125508    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125510    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125512    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125515    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125518    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125520    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125522    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125525    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125528    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125530    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125533    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125535    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125539    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125542    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125545    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125548    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125550    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:24.128304 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125553    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125555    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125558    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125560    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125563    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125565    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125567    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125570    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125572    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125574    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125577    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125579    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125582    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125584    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125586    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125589    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125591    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125593    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125597    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125600    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:24.128826 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125603    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125605    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125608    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125610    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125613    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125615    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125618    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125621    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125623    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125626    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125629    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125632    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125634    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125636    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.125641    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:44:24.129312 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125773    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125779    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125781    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125784    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125787    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125790    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125793    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125795    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125798    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125801    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125803    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125806    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125809    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125811    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125814    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125816    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125819    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125821    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125823    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125826    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:24.129719 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125828    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125831    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125833    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125836    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125838    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125841    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125843    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125846    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125849    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125851    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125854    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125856    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125859    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125861    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125864    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125866    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125868    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125871    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125873    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125876    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:24.130177 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125878    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125881    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125883    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125885    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125888    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125890    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125893    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125895    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125898    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125900    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125902    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125905    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125908    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125910    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125913    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125915    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125918    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125920    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125923    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:24.130650 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125926    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125929    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125932    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125935    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125937    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125940    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125942    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125944    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125947    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125949    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125952    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125956    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125960    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125963    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125966    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125969    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125972    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125975    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125977    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:24.131109 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125980    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125983    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125985    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125988    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125990    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125993    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125995    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.125997    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126000    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126002    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126005    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126007    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126011    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126014    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126017    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126020    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126023    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126026    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126029    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:24.131603 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126032    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:24.132169 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:24.126035    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:24.132169 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.126040    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:44:24.132169 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.126172    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 18:44:24.132169 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.128800    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 18:44:24.132169 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.129924    2581 server.go:1019] "Starting client certificate rotation"
Sep 18 18:44:24.132169 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.130029    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:44:24.132169 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.130057    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:44:24.134865 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.134838    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:44:24.137051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.137024    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:44:24.148968 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.148941    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 18 18:44:24.150958 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.150944    2581 log.go:25] "Validated CRI v1 image API"
Sep 18 18:44:24.151784 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.151766    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 18:44:24.154477 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.154456    2581 fs.go:135] Filesystem UUIDs: map[37671d48-791c-400f-a32d-ed4321a40031:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bfeb24be-0798-49bf-8af1-b86e791530af:/dev/nvme0n1p4]
Sep 18 18:44:24.154551 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.154477    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 18:44:24.160349 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.160221    2581 manager.go:216] Machine: {Timestamp:2026-09-18 18:44:24.158788472 +0000 UTC m=+0.377796040 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3191714 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20400fc75748a52b6dc9820c391a07 SystemUUID:ec20400f-c757-48a5-2b6d-c9820c391a07 BootID:a922ea38-5787-48a0-be89-34f967785cea Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f8:0c:3d:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f8:0c:3d:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:2c:21:44:a5:7a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 18:44:24.160349 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.160341    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 18:44:24.160463 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.160444    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 18:44:24.164277 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.164240    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 18:44:24.164439 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.164283    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-52.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 18:44:24.165142 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.165132    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 18:44:24.165181 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.165144    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 18:44:24.165181 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.165153    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:44:24.165181 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.165167    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:44:24.165265 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.165252    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 18:44:24.169109 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.169090    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:44:24.170094 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.170081    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:44:24.170222 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.170213    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 18:44:24.170250 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.170243    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 18:44:24.170276 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.170253    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 18:44:24.170276 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.170271    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 18:44:24.170332 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.170283    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 18 18:44:24.170332 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.170305    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 18:44:24.170894 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.170881    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:44:24.170955 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.170899    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:44:24.173437 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.173413    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 18:44:24.176605 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.176590    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 18:44:24.176677 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.176621    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 18:44:24.178698 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.178681    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 18:44:24.178698 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.178715    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 18:44:24.178807 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.178721    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 18:44:24.178807 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.178726    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 18:44:24.178807 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.178735    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 18:44:24.179023 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.179009    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 18:44:24.179082 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.179043    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 18:44:24.179082 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.179055    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 18:44:24.179082 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.179067    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 18:44:24.179082 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.179079    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 18:44:24.179223 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.179099    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 18:44:24.179223 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.179109    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 18:44:24.180060 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.180042    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 18:44:24.180130 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.180065    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 18:44:24.184489 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.184470    2581 server.go:1268] "Started kubelet"
Sep 18 18:44:24.184611 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.184534    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 18:44:24.184654 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.184568    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 18:44:24.184654 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.184649    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 18:44:24.184828 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.184805    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-52.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 18:44:24.184981 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.184953    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-52.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 18:44:24.185085 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.185037    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 18:44:24.185328 ip-10-0-136-52 systemd[1]: Started Kubernetes Kubelet.
Sep 18 18:44:24.186635 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.186448    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 18:44:24.186635 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.186634    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 18:44:24.189768 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.189746    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 18:44:24.196305 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.195154    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-52.ec2.internal.18d67e913cf567a9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-52.ec2.internal,UID:ip-10-0-136-52.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-52.ec2.internal,},FirstTimestamp:2026-09-18 18:44:24.184424361 +0000 UTC m=+0.403431930,LastTimestamp:2026-09-18 18:44:24.184424361 +0000 UTC m=+0.403431930,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-52.ec2.internal,}"
Sep 18 18:44:24.199853 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.199829    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 18:44:24.199969 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.199852    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 18:44:24.200039 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.199965    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:24.200603 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.200586    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 18:44:24.200688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.200607    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 18:44:24.200688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.200588    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 18:44:24.200688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.200675    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 18:44:24.200688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.200685    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 18:44:24.201068 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.201050    2581 factory.go:55] Registering systemd factory
Sep 18 18:44:24.201140 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.201073    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 18 18:44:24.201287 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.201274    2581 factory.go:153] Registering CRI-O factory
Sep 18 18:44:24.201287 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.201288    2581 factory.go:223] Registration of the crio container factory successfully
Sep 18 18:44:24.201422 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.201357    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 18:44:24.201422 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.201377    2581 factory.go:103] Registering Raw factory
Sep 18 18:44:24.201422 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.201393    2581 manager.go:1196] Started watching for new ooms in manager
Sep 18 18:44:24.201852 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.201829    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 18:44:24.201944 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.201884    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-52.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 18:44:24.201982 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.201944    2581 manager.go:349] Starting recovery of all containers
Sep 18 18:44:24.212000 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.211979    2581 manager.go:354] Recovery completed
Sep 18 18:44:24.218463 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.218440    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:24.220936 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.220914    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:24.221025 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.220951    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:24.221025 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.220962    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:24.221581 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.221562    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 18:44:24.221581 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.221579    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 18:44:24.221726 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.221596    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:44:24.221726 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.221566    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:24.222327 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.222312    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:24.222397 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.222345    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:24.222397 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.222360    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:24.223896 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.223882    2581 policy_none.go:49] "None policy: Start"
Sep 18 18:44:24.223896 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.223899    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 18:44:24.223993 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.223909    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 18:44:24.225932 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.225915    2581 policy_none.go:47] "Start"
Sep 18 18:44:24.231246 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.231225    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cdhf8"
Sep 18 18:44:24.232237 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.232162    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-52.ec2.internal.18d67e913f228d85  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-52.ec2.internal,UID:ip-10-0-136-52.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-52.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-52.ec2.internal,},FirstTimestamp:2026-09-18 18:44:24.220937605 +0000 UTC m=+0.439945173,LastTimestamp:2026-09-18 18:44:24.220937605 +0000 UTC m=+0.439945173,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-52.ec2.internal,}"
Sep 18 18:44:24.240153 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.240125    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cdhf8"
Sep 18 18:44:24.246153 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.246066    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-52.ec2.internal.18d67e913f22d8ed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-52.ec2.internal,UID:ip-10-0-136-52.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-52.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-52.ec2.internal,},FirstTimestamp:2026-09-18 18:44:24.220956909 +0000 UTC m=+0.439964477,LastTimestamp:2026-09-18 18:44:24.220956909 +0000 UTC m=+0.439964477,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-52.ec2.internal,}"
Sep 18 18:44:24.263477 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.263456    2581 manager.go:337] "Starting Device Plugin manager"
Sep 18 18:44:24.281180 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.263518    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 18:44:24.281180 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.263532    2581 server.go:87] "Starting device plugin registration server"
Sep 18 18:44:24.281180 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.263916    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 18:44:24.281180 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.263929    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 18:44:24.281180 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.264012    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 18:44:24.281180 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.264092    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 18:44:24.281180 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.264100    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 18:44:24.281180 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.265119    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 18:44:24.281180 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.265164    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:24.304921 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.304881    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 18:44:24.306200 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.306183    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 18:44:24.306271 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.306231    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 18:44:24.306271 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.306255    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 18:44:24.306353 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.306293    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 18:44:24.308766 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.308743    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:24.365084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.364957    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:24.365967 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.365950    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:24.366032 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.365980    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:24.366032 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.365990    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:24.366032 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.366012    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.375902 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.375873    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.375902 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.375901    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-52.ec2.internal\": node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:24.405785 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.405756    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:24.407106 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.407086    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal"]
Sep 18 18:44:24.407180 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.407157    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:24.408862 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.408848    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:24.408926 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.408881    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:24.408926 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.408890    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:24.411201 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.411184    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:24.411344 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.411321    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.411421 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.411366    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:24.412053 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.412037    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:24.412134 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.412072    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:24.412134 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.412087    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:24.412238 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.412073    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:24.412238 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.412227    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:24.412238 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.412238    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:24.414627 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.414608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.414760 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.414642    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:24.415857 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.415841    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:24.415968 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.415872    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:24.415968 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.415887    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:24.434673 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.434644    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-52.ec2.internal\" not found" node="ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.439445 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.439424    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-52.ec2.internal\" not found" node="ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.502106 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.502068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09bfe3eb0f63920fa5521f093ea0db2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal\" (UID: \"09bfe3eb0f63920fa5521f093ea0db2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.502106 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.502100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09bfe3eb0f63920fa5521f093ea0db2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal\" (UID: \"09bfe3eb0f63920fa5521f093ea0db2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.502304 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.502124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c29e46335904b8ccc50d8f973acddf9a-config\") pod \"kube-apiserver-proxy-ip-10-0-136-52.ec2.internal\" (UID: \"c29e46335904b8ccc50d8f973acddf9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.505872 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.505852    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:24.602764 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.602725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09bfe3eb0f63920fa5521f093ea0db2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal\" (UID: \"09bfe3eb0f63920fa5521f093ea0db2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.602764 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.602760    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09bfe3eb0f63920fa5521f093ea0db2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal\" (UID: \"09bfe3eb0f63920fa5521f093ea0db2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.602961 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.602776    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09bfe3eb0f63920fa5521f093ea0db2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal\" (UID: \"09bfe3eb0f63920fa5521f093ea0db2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.602961 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.602813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c29e46335904b8ccc50d8f973acddf9a-config\") pod \"kube-apiserver-proxy-ip-10-0-136-52.ec2.internal\" (UID: \"c29e46335904b8ccc50d8f973acddf9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.602961 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.602783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c29e46335904b8ccc50d8f973acddf9a-config\") pod \"kube-apiserver-proxy-ip-10-0-136-52.ec2.internal\" (UID: \"c29e46335904b8ccc50d8f973acddf9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.602961 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.602858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09bfe3eb0f63920fa5521f093ea0db2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal\" (UID: \"09bfe3eb0f63920fa5521f093ea0db2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.606793 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.606776    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:24.707743 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.707670    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:24.738849 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.738827    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.742982 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:24.742960    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:24.807841 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.807798    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:24.908359 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:24.908328    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:25.009019 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:25.008940    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:25.094023 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.093988    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:25.109285 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:25.109252    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:25.129452 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.129424    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 18:44:25.129633 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.129609    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:44:25.129682 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.129609    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:44:25.210452 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:25.210253    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:25.247145 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.247100    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 18:39:23 +0000 UTC" deadline="2028-04-22 16:04:57.298026995 +0000 UTC"
Sep 18 18:44:25.247145 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.247138    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13965h20m32.050892061s"
Sep 18 18:44:25.310840 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:25.310763    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-52.ec2.internal\" not found"
Sep 18 18:44:25.332203 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.332172    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:25.400333 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.400294    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:25.427354 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.427325    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:44:25.428282 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.428263    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal"
Sep 18 18:44:25.453873 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.453842    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:44:25.605280 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:25.605250    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29e46335904b8ccc50d8f973acddf9a.slice/crio-f74f110713aa60df34483c2a2dbff64236123cd815d241c8c6b3529296b6e219 WatchSource:0}: Error finding container f74f110713aa60df34483c2a2dbff64236123cd815d241c8c6b3529296b6e219: Status 404 returned error can't find the container with id f74f110713aa60df34483c2a2dbff64236123cd815d241c8c6b3529296b6e219
Sep 18 18:44:25.605561 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:25.605539    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09bfe3eb0f63920fa5521f093ea0db2b.slice/crio-4822cedf548ed15c9646a707b953dabf97bb2b724f4d8194720b278ad1f4d247 WatchSource:0}: Error finding container 4822cedf548ed15c9646a707b953dabf97bb2b724f4d8194720b278ad1f4d247: Status 404 returned error can't find the container with id 4822cedf548ed15c9646a707b953dabf97bb2b724f4d8194720b278ad1f4d247
Sep 18 18:44:25.610873 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.610854    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:44:25.665320 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:25.665291    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:26.171343 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.171308    2581 apiserver.go:52] "Watching apiserver"
Sep 18 18:44:26.179889 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.179854    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 18:44:26.181294 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.181267    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-bspq5","openshift-image-registry/node-ca-rqb8l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal","openshift-multus/multus-additional-cni-plugins-twwkm","openshift-multus/multus-j5f5r","openshift-multus/network-metrics-daemon-skwt5","kube-system/global-pull-secret-syncer-zsrbq","kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal","openshift-cluster-node-tuning-operator/tuned-j2n6t","openshift-network-diagnostics/network-check-target-9hxwx","openshift-network-operator/iptables-alerter-469b7","openshift-ovn-kubernetes/ovnkube-node-86j9s","kube-system/konnectivity-agent-bxzsl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"]
Sep 18 18:44:26.183624 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.183597    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:26.183754 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.183690    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:26.185870 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.185851    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:26.187289 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.187270    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:26.187816 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.187795    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.189671 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.189651    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-68xzp\""
Sep 18 18:44:26.189777 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.189763    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 18:44:26.190051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.190033    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 18:44:26.190936 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.190895    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.191033 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.190941    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 18:44:26.191139 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.191038    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jrxx7\""
Sep 18 18:44:26.191139 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.191114    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 18:44:26.191271 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.191256    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 18:44:26.193018 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.192996    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.195569 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.195544    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:26.195650 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.195601    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:26.195919 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.195902    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 18:44:26.196090 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.196075    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 18:44:26.196731 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.196716    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 18:44:26.197259 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.197243    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.198914 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.198895    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 18:44:26.199048 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.199030    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 18:44:26.199188 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.199172    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wfw6r\""
Sep 18 18:44:26.199441 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.199424    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 18:44:26.199501 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.199424    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d2gf9\""
Sep 18 18:44:26.199611 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.199598    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.199769 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.199754    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:44:26.199954 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.199929    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vp66d\""
Sep 18 18:44:26.200374 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.200357    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 18:44:26.200767 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.200753    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 18:44:26.201869 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.201598    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.203450 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.203432    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 18:44:26.203931 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.203880    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 18:44:26.204198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.204181    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 18:44:26.204422 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.204410    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 18:44:26.204553 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.204538    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dj59h\""
Sep 18 18:44:26.208188 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.208167    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ctjn9\""
Sep 18 18:44:26.208457 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.208441    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 18:44:26.209597 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.209548    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:26.209690 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.209637    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:26.210280 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.210259    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd7d39c2-89db-45d7-81b7-9ff9c7290424-system-cni-dir\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.210371 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.210297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-device-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.210371 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.210332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-socket-dir-parent\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.210371 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.210360    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-run-k8s-cni-cncf-io\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.210514 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.210384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-run-multus-certs\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.210514 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.210500    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211228    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.210410    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m6rv\" (UniqueName: \"kubernetes.io/projected/2415ad06-8ee8-4004-a147-65f2fe315aee-kube-api-access-4m6rv\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211440    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-cni-binary-copy\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-run-netns\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-var-lib-kubelet\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-modprobe-d\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd7d39c2-89db-45d7-81b7-9ff9c7290424-tuning-conf-dir\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5swt\" (UniqueName: \"kubernetes.io/projected/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-kube-api-access-b5swt\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d2f53f4b-0b86-4f9a-a5f6-0c428539290f-hosts-file\") pod \"node-resolver-bspq5\" (UID: \"d2f53f4b-0b86-4f9a-a5f6-0c428539290f\") " pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2f53f4b-0b86-4f9a-a5f6-0c428539290f-tmp-dir\") pod \"node-resolver-bspq5\" (UID: \"d2f53f4b-0b86-4f9a-a5f6-0c428539290f\") " pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211788    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa308550-2945-40b0-ba91-87a9c8541846-host\") pod \"node-ca-rqb8l\" (UID: \"aa308550-2945-40b0-ba91-87a9c8541846\") " pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211812    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxlvf\" (UniqueName: \"kubernetes.io/projected/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-kube-api-access-bxlvf\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-conf-dir\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b7z6\" (UniqueName: \"kubernetes.io/projected/d2f53f4b-0b86-4f9a-a5f6-0c428539290f-kube-api-access-8b7z6\") pod \"node-resolver-bspq5\" (UID: \"d2f53f4b-0b86-4f9a-a5f6-0c428539290f\") " pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211933    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-var-lib-cni-bin\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.212146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.211988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-etc-kubernetes\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.212038    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-run\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.212073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-dbus\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.212105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-system-cni-dir\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.212151    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-tuned\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.212211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.212251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.212281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-hostroot\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.212336    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-host\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.212360    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lchs5\" (UniqueName: \"kubernetes.io/projected/6d1247b6-0d71-4c51-93b2-0f6407e179d6-kube-api-access-lchs5\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213156    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cab91e29-3ede-43e0-9209-0945d73869a7-konnectivity-ca\") pod \"konnectivity-agent-bxzsl\" (UID: \"cab91e29-3ede-43e0-9209-0945d73869a7\") " pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-socket-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213283    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-registration-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213372    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5np7p\" (UniqueName: \"kubernetes.io/projected/aa308550-2945-40b0-ba91-87a9c8541846-kube-api-access-5np7p\") pod \"node-ca-rqb8l\" (UID: \"aa308550-2945-40b0-ba91-87a9c8541846\") " pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.213465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd7d39c2-89db-45d7-81b7-9ff9c7290424-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-kubernetes\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213494    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-cni-dir\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-sysconfig\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-sys\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd7d39c2-89db-45d7-81b7-9ff9c7290424-cnibin\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.213974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-sys-fs\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214006    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd7d39c2-89db-45d7-81b7-9ff9c7290424-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-systemd\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47v78\" (UniqueName: \"kubernetes.io/projected/dd7d39c2-89db-45d7-81b7-9ff9c7290424-kube-api-access-47v78\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d1247b6-0d71-4c51-93b2-0f6407e179d6-tmp\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-kubelet-config\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214187    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-cnibin\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-sysctl-d\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-lib-modules\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.217158 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd7d39c2-89db-45d7-81b7-9ff9c7290424-cni-binary-copy\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-daemon-config\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214280    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-var-lib-kubelet\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214304    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cab91e29-3ede-43e0-9209-0945d73869a7-agent-certs\") pod \"konnectivity-agent-bxzsl\" (UID: \"cab91e29-3ede-43e0-9209-0945d73869a7\") " pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aa308550-2945-40b0-ba91-87a9c8541846-serviceca\") pod \"node-ca-rqb8l\" (UID: \"aa308550-2945-40b0-ba91-87a9c8541846\") " pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-var-lib-cni-multus\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214488    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-sysctl-conf\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214638    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214728    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.214848    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.215122    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.215216    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.215275    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wfggf\""
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.215597    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bgjfn\""
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.215754    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.217903    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 18:44:26.218076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.218011    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 18:44:26.218526 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.218124    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 18:44:26.224025 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.223573    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:44:26.257845 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.257819    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fg2zc"
Sep 18 18:44:26.268128 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.268101    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fg2zc"
Sep 18 18:44:26.302391 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.302323    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 18:44:26.312422 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.312320    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal" event={"ID":"c29e46335904b8ccc50d8f973acddf9a","Type":"ContainerStarted","Data":"f74f110713aa60df34483c2a2dbff64236123cd815d241c8c6b3529296b6e219"}
Sep 18 18:44:26.315242 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:26.315380 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.315380 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315277    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-hostroot\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.315380 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-host\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.315380 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315318    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lchs5\" (UniqueName: \"kubernetes.io/projected/6d1247b6-0d71-4c51-93b2-0f6407e179d6-kube-api-access-lchs5\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.315380 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skwst\" (UniqueName: \"kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst\") pod \"network-check-target-9hxwx\" (UID: \"9a781944-d73f-4c37-8c50-8c71aaa9b2a6\") " pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:26.315380 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb0643b9-b435-4bc5-be37-8be315df6412-env-overrides\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cab91e29-3ede-43e0-9209-0945d73869a7-konnectivity-ca\") pod \"konnectivity-agent-bxzsl\" (UID: \"cab91e29-3ede-43e0-9209-0945d73869a7\") " pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315415    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-socket-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-registration-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315435    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal" event={"ID":"09bfe3eb0f63920fa5521f093ea0db2b","Type":"ContainerStarted","Data":"4822cedf548ed15c9646a707b953dabf97bb2b724f4d8194720b278ad1f4d247"}
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5np7p\" (UniqueName: \"kubernetes.io/projected/aa308550-2945-40b0-ba91-87a9c8541846-kube-api-access-5np7p\") pod \"node-ca-rqb8l\" (UID: \"aa308550-2945-40b0-ba91-87a9c8541846\") " pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd7d39c2-89db-45d7-81b7-9ff9c7290424-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315500    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-kubernetes\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-cni-dir\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-sysconfig\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-sys\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-systemd-units\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315620    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-cni-netd\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.315638    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:26.315679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd7d39c2-89db-45d7-81b7-9ff9c7290424-cnibin\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-sys-fs\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd7d39c2-89db-45d7-81b7-9ff9c7290424-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-systemd\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47v78\" (UniqueName: \"kubernetes.io/projected/dd7d39c2-89db-45d7-81b7-9ff9c7290424-kube-api-access-47v78\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315843    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315851    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d1247b6-0d71-4c51-93b2-0f6407e179d6-tmp\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb0643b9-b435-4bc5-be37-8be315df6412-ovnkube-script-lib\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-kubelet-config\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-cnibin\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-sysctl-d\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.315999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-lib-modules\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd7d39c2-89db-45d7-81b7-9ff9c7290424-cni-binary-copy\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316071    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-daemon-config\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-var-lib-kubelet\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-etc-openvswitch\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.316373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cab91e29-3ede-43e0-9209-0945d73869a7-agent-certs\") pod \"konnectivity-agent-bxzsl\" (UID: \"cab91e29-3ede-43e0-9209-0945d73869a7\") " pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aa308550-2945-40b0-ba91-87a9c8541846-serviceca\") pod \"node-ca-rqb8l\" (UID: \"aa308550-2945-40b0-ba91-87a9c8541846\") " pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-var-lib-cni-multus\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316165    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-run-ovn\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-sysctl-conf\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd7d39c2-89db-45d7-81b7-9ff9c7290424-system-cni-dir\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-device-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-socket-dir-parent\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316331    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-run-k8s-cni-cncf-io\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-run-multus-certs\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316368    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316435    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-hostroot\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316522    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-host\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316762    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cab91e29-3ede-43e0-9209-0945d73869a7-konnectivity-ca\") pod \"konnectivity-agent-bxzsl\" (UID: \"cab91e29-3ede-43e0-9209-0945d73869a7\") " pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-kubelet-config\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-cnibin\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316872    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-socket-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316911    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-registration-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.317051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-lib-modules\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.316383    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-var-lib-openvswitch\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317240    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5wb5\" (UniqueName: \"kubernetes.io/projected/fb0643b9-b435-4bc5-be37-8be315df6412-kube-api-access-x5wb5\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m6rv\" (UniqueName: \"kubernetes.io/projected/2415ad06-8ee8-4004-a147-65f2fe315aee-kube-api-access-4m6rv\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-cni-binary-copy\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-run-netns\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317342    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-var-lib-kubelet\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-modprobe-d\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317385    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3cc0af90-41b6-4637-b38f-ee10ffc79837-host-slash\") pod \"iptables-alerter-469b7\" (UID: \"3cc0af90-41b6-4637-b38f-ee10ffc79837\") " pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317421    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-run-netns\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317455    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd7d39c2-89db-45d7-81b7-9ff9c7290424-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd7d39c2-89db-45d7-81b7-9ff9c7290424-tuning-conf-dir\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317508    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-sysconfig\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-sys-fs\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317530    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317543    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd7d39c2-89db-45d7-81b7-9ff9c7290424-cni-binary-copy\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.317834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317563    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-sys\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-cni-dir\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317456    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd7d39c2-89db-45d7-81b7-9ff9c7290424-cnibin\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-var-lib-cni-multus\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317919    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-kubernetes\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317955    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-var-lib-kubelet\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2415ad06-8ee8-4004-a147-65f2fe315aee-device-dir\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318057    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd7d39c2-89db-45d7-81b7-9ff9c7290424-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-daemon-config\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.317855    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-systemd\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-run-multus-certs\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-cni-binary-copy\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318231    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-run-k8s-cni-cncf-io\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-socket-dir-parent\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318264    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-run-netns\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318271    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd7d39c2-89db-45d7-81b7-9ff9c7290424-system-cni-dir\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318273    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-sysctl-conf\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-var-lib-kubelet\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.318714 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318334    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aa308550-2945-40b0-ba91-87a9c8541846-serviceca\") pod \"node-ca-rqb8l\" (UID: \"aa308550-2945-40b0-ba91-87a9c8541846\") " pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.318344    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.318372    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret podName:1dcfcf8b-4bd5-4919-9315-26fe6852ba7d nodeName:}" failed. No retries permitted until 2026-09-18 18:44:26.818354767 +0000 UTC m=+3.037362324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret") pod "global-pull-secret-syncer-zsrbq" (UID: "1dcfcf8b-4bd5-4919-9315-26fe6852ba7d") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318383    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-modprobe-d\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5jxx\" (UniqueName: \"kubernetes.io/projected/3cc0af90-41b6-4637-b38f-ee10ffc79837-kube-api-access-x5jxx\") pod \"iptables-alerter-469b7\" (UID: \"3cc0af90-41b6-4637-b38f-ee10ffc79837\") " pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-sysctl-d\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd7d39c2-89db-45d7-81b7-9ff9c7290424-tuning-conf-dir\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.318468    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs podName:85ea9986-0a9c-4b8a-835b-f42a46dd8e55 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:26.818449559 +0000 UTC m=+3.037457123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs") pod "network-metrics-daemon-skwt5" (UID: "85ea9986-0a9c-4b8a-835b-f42a46dd8e55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-log-socket\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318528    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-run-ovn-kubernetes\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb0643b9-b435-4bc5-be37-8be315df6412-ovn-node-metrics-cert\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5swt\" (UniqueName: \"kubernetes.io/projected/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-kube-api-access-b5swt\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d2f53f4b-0b86-4f9a-a5f6-0c428539290f-hosts-file\") pod \"node-resolver-bspq5\" (UID: \"d2f53f4b-0b86-4f9a-a5f6-0c428539290f\") " pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318687    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d2f53f4b-0b86-4f9a-a5f6-0c428539290f-hosts-file\") pod \"node-resolver-bspq5\" (UID: \"d2f53f4b-0b86-4f9a-a5f6-0c428539290f\") " pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2f53f4b-0b86-4f9a-a5f6-0c428539290f-tmp-dir\") pod \"node-resolver-bspq5\" (UID: \"d2f53f4b-0b86-4f9a-a5f6-0c428539290f\") " pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318807    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa308550-2945-40b0-ba91-87a9c8541846-host\") pod \"node-ca-rqb8l\" (UID: \"aa308550-2945-40b0-ba91-87a9c8541846\") " pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.319502 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxlvf\" (UniqueName: \"kubernetes.io/projected/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-kube-api-access-bxlvf\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318856    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3cc0af90-41b6-4637-b38f-ee10ffc79837-iptables-alerter-script\") pod \"iptables-alerter-469b7\" (UID: \"3cc0af90-41b6-4637-b38f-ee10ffc79837\") " pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318878    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-run-systemd\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-run-openvswitch\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318921    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-conf-dir\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318941    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-node-log\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-cni-bin\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318983    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb0643b9-b435-4bc5-be37-8be315df6412-ovnkube-config\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b7z6\" (UniqueName: \"kubernetes.io/projected/d2f53f4b-0b86-4f9a-a5f6-0c428539290f-kube-api-access-8b7z6\") pod \"node-resolver-bspq5\" (UID: \"d2f53f4b-0b86-4f9a-a5f6-0c428539290f\") " pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319027    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa308550-2945-40b0-ba91-87a9c8541846-host\") pod \"node-ca-rqb8l\" (UID: \"aa308550-2945-40b0-ba91-87a9c8541846\") " pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319029    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-var-lib-cni-bin\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319063    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-host-var-lib-cni-bin\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319062    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-etc-kubernetes\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-etc-kubernetes\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-multus-conf-dir\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319134    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-run\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d1247b6-0d71-4c51-93b2-0f6407e179d6-run\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.320345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-kubelet\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319204    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-dbus\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319275    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-system-cni-dir\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319296    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-tuned\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.318983    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2f53f4b-0b86-4f9a-a5f6-0c428539290f-tmp-dir\") pod \"node-resolver-bspq5\" (UID: \"d2f53f4b-0b86-4f9a-a5f6-0c428539290f\") " pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-slash\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319400    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-dbus\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.319444    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-system-cni-dir\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.320535    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d1247b6-0d71-4c51-93b2-0f6407e179d6-tmp\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.322009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cab91e29-3ede-43e0-9209-0945d73869a7-agent-certs\") pod \"konnectivity-agent-bxzsl\" (UID: \"cab91e29-3ede-43e0-9209-0945d73869a7\") " pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:26.322217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.322180    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d1247b6-0d71-4c51-93b2-0f6407e179d6-etc-tuned\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.327569 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.327542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5np7p\" (UniqueName: \"kubernetes.io/projected/aa308550-2945-40b0-ba91-87a9c8541846-kube-api-access-5np7p\") pod \"node-ca-rqb8l\" (UID: \"aa308550-2945-40b0-ba91-87a9c8541846\") " pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.328939 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.328907    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47v78\" (UniqueName: \"kubernetes.io/projected/dd7d39c2-89db-45d7-81b7-9ff9c7290424-kube-api-access-47v78\") pod \"multus-additional-cni-plugins-twwkm\" (UID: \"dd7d39c2-89db-45d7-81b7-9ff9c7290424\") " pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.329909 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.329834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lchs5\" (UniqueName: \"kubernetes.io/projected/6d1247b6-0d71-4c51-93b2-0f6407e179d6-kube-api-access-lchs5\") pod \"tuned-j2n6t\" (UID: \"6d1247b6-0d71-4c51-93b2-0f6407e179d6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.331128 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.331099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m6rv\" (UniqueName: \"kubernetes.io/projected/2415ad06-8ee8-4004-a147-65f2fe315aee-kube-api-access-4m6rv\") pod \"aws-ebs-csi-driver-node-k6l57\" (UID: \"2415ad06-8ee8-4004-a147-65f2fe315aee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.331393 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.331377    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxlvf\" (UniqueName: \"kubernetes.io/projected/687f69e1-ac76-4d49-9a4d-c829f3d24f4e-kube-api-access-bxlvf\") pod \"multus-j5f5r\" (UID: \"687f69e1-ac76-4d49-9a4d-c829f3d24f4e\") " pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.332020 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.331978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b7z6\" (UniqueName: \"kubernetes.io/projected/d2f53f4b-0b86-4f9a-a5f6-0c428539290f-kube-api-access-8b7z6\") pod \"node-resolver-bspq5\" (UID: \"d2f53f4b-0b86-4f9a-a5f6-0c428539290f\") " pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.333308 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.333289    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5swt\" (UniqueName: \"kubernetes.io/projected/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-kube-api-access-b5swt\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:26.420000 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.419964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-etc-openvswitch\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420166 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-run-ovn\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420166 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420057    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-etc-openvswitch\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420166 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420077    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-var-lib-openvswitch\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420166 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wb5\" (UniqueName: \"kubernetes.io/projected/fb0643b9-b435-4bc5-be37-8be315df6412-kube-api-access-x5wb5\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420166 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420125    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3cc0af90-41b6-4637-b38f-ee10ffc79837-host-slash\") pod \"iptables-alerter-469b7\" (UID: \"3cc0af90-41b6-4637-b38f-ee10ffc79837\") " pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.420166 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420123    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-run-ovn\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420166 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420146    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-run-netns\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420177    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-var-lib-openvswitch\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-run-netns\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3cc0af90-41b6-4637-b38f-ee10ffc79837-host-slash\") pod \"iptables-alerter-469b7\" (UID: \"3cc0af90-41b6-4637-b38f-ee10ffc79837\") " pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5jxx\" (UniqueName: \"kubernetes.io/projected/3cc0af90-41b6-4637-b38f-ee10ffc79837-kube-api-access-x5jxx\") pod \"iptables-alerter-469b7\" (UID: \"3cc0af90-41b6-4637-b38f-ee10ffc79837\") " pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-log-socket\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-run-ovn-kubernetes\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb0643b9-b435-4bc5-be37-8be315df6412-ovn-node-metrics-cert\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420299    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-log-socket\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420323    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-run-ovn-kubernetes\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3cc0af90-41b6-4637-b38f-ee10ffc79837-iptables-alerter-script\") pod \"iptables-alerter-469b7\" (UID: \"3cc0af90-41b6-4637-b38f-ee10ffc79837\") " pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-run-systemd\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-run-openvswitch\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420458    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-run-systemd\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-run-openvswitch\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-node-log\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-node-log\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.420546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-cni-bin\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420574    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb0643b9-b435-4bc5-be37-8be315df6412-ovnkube-config\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420601    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-kubelet\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-cni-bin\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420624    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-kubelet\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-slash\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420669    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skwst\" (UniqueName: \"kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst\") pod \"network-check-target-9hxwx\" (UID: \"9a781944-d73f-4c37-8c50-8c71aaa9b2a6\") " pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb0643b9-b435-4bc5-be37-8be315df6412-env-overrides\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-systemd-units\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-cni-netd\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420815    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb0643b9-b435-4bc5-be37-8be315df6412-ovnkube-script-lib\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-systemd-units\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420736    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-slash\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3cc0af90-41b6-4637-b38f-ee10ffc79837-iptables-alerter-script\") pod \"iptables-alerter-469b7\" (UID: \"3cc0af90-41b6-4637-b38f-ee10ffc79837\") " pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420958    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb0643b9-b435-4bc5-be37-8be315df6412-host-cni-netd\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.420975    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb0643b9-b435-4bc5-be37-8be315df6412-ovnkube-config\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421589 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.421047    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb0643b9-b435-4bc5-be37-8be315df6412-env-overrides\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.421589 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.421271    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb0643b9-b435-4bc5-be37-8be315df6412-ovnkube-script-lib\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.423339 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.423318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb0643b9-b435-4bc5-be37-8be315df6412-ovn-node-metrics-cert\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.431477 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.431450    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:26.431477 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.431482    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:26.431675 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.431495    2581 projected.go:196] Error preparing data for projected volume kube-api-access-skwst for pod openshift-network-diagnostics/network-check-target-9hxwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:26.431675 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.431555    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst podName:9a781944-d73f-4c37-8c50-8c71aaa9b2a6 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:26.931537523 +0000 UTC m=+3.150545080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-skwst" (UniqueName: "kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst") pod "network-check-target-9hxwx" (UID: "9a781944-d73f-4c37-8c50-8c71aaa9b2a6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:26.434573 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.434544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5jxx\" (UniqueName: \"kubernetes.io/projected/3cc0af90-41b6-4637-b38f-ee10ffc79837-kube-api-access-x5jxx\") pod \"iptables-alerter-469b7\" (UID: \"3cc0af90-41b6-4637-b38f-ee10ffc79837\") " pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.439008 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.438981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5wb5\" (UniqueName: \"kubernetes.io/projected/fb0643b9-b435-4bc5-be37-8be315df6412-kube-api-access-x5wb5\") pod \"ovnkube-node-86j9s\" (UID: \"fb0643b9-b435-4bc5-be37-8be315df6412\") " pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.499663 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.499633    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:26.513893 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.513871    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57"
Sep 18 18:44:26.523354 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:26.523327    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2415ad06_8ee8_4004_a147_65f2fe315aee.slice/crio-ff1cf8d35f197c8b33f7151e09c46e731ee99f83ef41aa98fe079b9e13a4e415 WatchSource:0}: Error finding container ff1cf8d35f197c8b33f7151e09c46e731ee99f83ef41aa98fe079b9e13a4e415: Status 404 returned error can't find the container with id ff1cf8d35f197c8b33f7151e09c46e731ee99f83ef41aa98fe079b9e13a4e415
Sep 18 18:44:26.524168 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.524152    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-twwkm"
Sep 18 18:44:26.529724 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.529682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j5f5r"
Sep 18 18:44:26.533508 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:26.533479    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd7d39c2_89db_45d7_81b7_9ff9c7290424.slice/crio-51968e330d906c8755b51d555631421241549f003f5e150eaed8f7b02102b09b WatchSource:0}: Error finding container 51968e330d906c8755b51d555631421241549f003f5e150eaed8f7b02102b09b: Status 404 returned error can't find the container with id 51968e330d906c8755b51d555631421241549f003f5e150eaed8f7b02102b09b
Sep 18 18:44:26.535496 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.535462    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j2n6t"
Sep 18 18:44:26.538611 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:26.538579    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod687f69e1_ac76_4d49_9a4d_c829f3d24f4e.slice/crio-24649bf4fa4e565eeb43cfd30f5c550389925cd3f4242e0149def7aa7f6a1cdf WatchSource:0}: Error finding container 24649bf4fa4e565eeb43cfd30f5c550389925cd3f4242e0149def7aa7f6a1cdf: Status 404 returned error can't find the container with id 24649bf4fa4e565eeb43cfd30f5c550389925cd3f4242e0149def7aa7f6a1cdf
Sep 18 18:44:26.543717 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.543536    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bspq5"
Sep 18 18:44:26.545095 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:26.545069    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1247b6_0d71_4c51_93b2_0f6407e179d6.slice/crio-2d6d873b3c2680a0c6cb7525c8fa1f8c4d8a7df5e029ee6a0fc03ff4ca4a15ae WatchSource:0}: Error finding container 2d6d873b3c2680a0c6cb7525c8fa1f8c4d8a7df5e029ee6a0fc03ff4ca4a15ae: Status 404 returned error can't find the container with id 2d6d873b3c2680a0c6cb7525c8fa1f8c4d8a7df5e029ee6a0fc03ff4ca4a15ae
Sep 18 18:44:26.549646 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.549629    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rqb8l"
Sep 18 18:44:26.553693 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:26.553670    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f53f4b_0b86_4f9a_a5f6_0c428539290f.slice/crio-1969153c46102636ed2f718d42088ae728e4e433dba5a6003d66c089c09f319c WatchSource:0}: Error finding container 1969153c46102636ed2f718d42088ae728e4e433dba5a6003d66c089c09f319c: Status 404 returned error can't find the container with id 1969153c46102636ed2f718d42088ae728e4e433dba5a6003d66c089c09f319c
Sep 18 18:44:26.558004 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.557978    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-469b7"
Sep 18 18:44:26.560001 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:26.559977    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa308550_2945_40b0_ba91_87a9c8541846.slice/crio-3a249f69428022d547b647d7777fc25c39583097dff9711ed63f81311125fe93 WatchSource:0}: Error finding container 3a249f69428022d547b647d7777fc25c39583097dff9711ed63f81311125fe93: Status 404 returned error can't find the container with id 3a249f69428022d547b647d7777fc25c39583097dff9711ed63f81311125fe93
Sep 18 18:44:26.563824 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.563798    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:26.569178 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:26.569148    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc0af90_41b6_4637_b38f_ee10ffc79837.slice/crio-b185238b969299251ea91b7c0ce393c827f085aebc8aa0a7634db5fc1dd4f6ef WatchSource:0}: Error finding container b185238b969299251ea91b7c0ce393c827f085aebc8aa0a7634db5fc1dd4f6ef: Status 404 returned error can't find the container with id b185238b969299251ea91b7c0ce393c827f085aebc8aa0a7634db5fc1dd4f6ef
Sep 18 18:44:26.576031 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:26.576003    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0643b9_b435_4bc5_be37_8be315df6412.slice/crio-3b825d1751b31e19e4f8e10393b63b2882248685300e816e29426685c288526c WatchSource:0}: Error finding container 3b825d1751b31e19e4f8e10393b63b2882248685300e816e29426685c288526c: Status 404 returned error can't find the container with id 3b825d1751b31e19e4f8e10393b63b2882248685300e816e29426685c288526c
Sep 18 18:44:26.823655 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.823571    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:26.823655 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:26.823618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:26.823854 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.823743    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:26.823854 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.823754    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:26.823854 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.823816    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret podName:1dcfcf8b-4bd5-4919-9315-26fe6852ba7d nodeName:}" failed. No retries permitted until 2026-09-18 18:44:27.823784399 +0000 UTC m=+4.042791970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret") pod "global-pull-secret-syncer-zsrbq" (UID: "1dcfcf8b-4bd5-4919-9315-26fe6852ba7d") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:26.823854 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:26.823829    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs podName:85ea9986-0a9c-4b8a-835b-f42a46dd8e55 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:27.823823368 +0000 UTC m=+4.042830922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs") pod "network-metrics-daemon-skwt5" (UID: "85ea9986-0a9c-4b8a-835b-f42a46dd8e55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:27.025157 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.025122    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skwst\" (UniqueName: \"kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst\") pod \"network-check-target-9hxwx\" (UID: \"9a781944-d73f-4c37-8c50-8c71aaa9b2a6\") " pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:27.025332 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:27.025310    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:27.025382 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:27.025340    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:27.025382 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:27.025378    2581 projected.go:196] Error preparing data for projected volume kube-api-access-skwst for pod openshift-network-diagnostics/network-check-target-9hxwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:27.025460 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:27.025444    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst podName:9a781944-d73f-4c37-8c50-8c71aaa9b2a6 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:28.025424586 +0000 UTC m=+4.244432152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-skwst" (UniqueName: "kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst") pod "network-check-target-9hxwx" (UID: "9a781944-d73f-4c37-8c50-8c71aaa9b2a6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:27.271407 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.271322    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 18:39:26 +0000 UTC" deadline="2028-06-27 01:56:42.238684244 +0000 UTC"
Sep 18 18:44:27.271407 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.271365    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15535h12m14.967324195s"
Sep 18 18:44:27.318338 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.318298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s" event={"ID":"fb0643b9-b435-4bc5-be37-8be315df6412","Type":"ContainerStarted","Data":"3b825d1751b31e19e4f8e10393b63b2882248685300e816e29426685c288526c"}
Sep 18 18:44:27.319670 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.319642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-469b7" event={"ID":"3cc0af90-41b6-4637-b38f-ee10ffc79837","Type":"ContainerStarted","Data":"b185238b969299251ea91b7c0ce393c827f085aebc8aa0a7634db5fc1dd4f6ef"}
Sep 18 18:44:27.320873 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.320828    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bspq5" event={"ID":"d2f53f4b-0b86-4f9a-a5f6-0c428539290f","Type":"ContainerStarted","Data":"1969153c46102636ed2f718d42088ae728e4e433dba5a6003d66c089c09f319c"}
Sep 18 18:44:27.322034 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.322009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j5f5r" event={"ID":"687f69e1-ac76-4d49-9a4d-c829f3d24f4e","Type":"ContainerStarted","Data":"24649bf4fa4e565eeb43cfd30f5c550389925cd3f4242e0149def7aa7f6a1cdf"}
Sep 18 18:44:27.323365 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.323315    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57" event={"ID":"2415ad06-8ee8-4004-a147-65f2fe315aee","Type":"ContainerStarted","Data":"ff1cf8d35f197c8b33f7151e09c46e731ee99f83ef41aa98fe079b9e13a4e415"}
Sep 18 18:44:27.324368 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.324339    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bxzsl" event={"ID":"cab91e29-3ede-43e0-9209-0945d73869a7","Type":"ContainerStarted","Data":"92daf8380e9f886dc01f473ed22d99e56af480e204d7a3c31005aa4b1f62bb9e"}
Sep 18 18:44:27.325485 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.325467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rqb8l" event={"ID":"aa308550-2945-40b0-ba91-87a9c8541846","Type":"ContainerStarted","Data":"3a249f69428022d547b647d7777fc25c39583097dff9711ed63f81311125fe93"}
Sep 18 18:44:27.326630 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.326607    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j2n6t" event={"ID":"6d1247b6-0d71-4c51-93b2-0f6407e179d6","Type":"ContainerStarted","Data":"2d6d873b3c2680a0c6cb7525c8fa1f8c4d8a7df5e029ee6a0fc03ff4ca4a15ae"}
Sep 18 18:44:27.327789 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.327751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-twwkm" event={"ID":"dd7d39c2-89db-45d7-81b7-9ff9c7290424","Type":"ContainerStarted","Data":"51968e330d906c8755b51d555631421241549f003f5e150eaed8f7b02102b09b"}
Sep 18 18:44:27.497404 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.497372    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:27.830763 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.830498    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:27.830763 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:27.830582    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:27.830763 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:27.830649    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:27.830763 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:27.830662    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:27.830763 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:27.830724    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs podName:85ea9986-0a9c-4b8a-835b-f42a46dd8e55 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:29.830692943 +0000 UTC m=+6.049700510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs") pod "network-metrics-daemon-skwt5" (UID: "85ea9986-0a9c-4b8a-835b-f42a46dd8e55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:27.830763 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:27.830740    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret podName:1dcfcf8b-4bd5-4919-9315-26fe6852ba7d nodeName:}" failed. No retries permitted until 2026-09-18 18:44:29.830731467 +0000 UTC m=+6.049739020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret") pod "global-pull-secret-syncer-zsrbq" (UID: "1dcfcf8b-4bd5-4919-9315-26fe6852ba7d") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:28.032776 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:28.032505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skwst\" (UniqueName: \"kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst\") pod \"network-check-target-9hxwx\" (UID: \"9a781944-d73f-4c37-8c50-8c71aaa9b2a6\") " pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:28.032776 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:28.032723    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:28.032776 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:28.032745    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:28.032776 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:28.032758    2581 projected.go:196] Error preparing data for projected volume kube-api-access-skwst for pod openshift-network-diagnostics/network-check-target-9hxwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:28.033089 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:28.032814    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst podName:9a781944-d73f-4c37-8c50-8c71aaa9b2a6 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:30.032797171 +0000 UTC m=+6.251804728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-skwst" (UniqueName: "kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst") pod "network-check-target-9hxwx" (UID: "9a781944-d73f-4c37-8c50-8c71aaa9b2a6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:28.272335 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:28.272272    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 18:39:26 +0000 UTC" deadline="2028-06-28 15:43:47.005220158 +0000 UTC"
Sep 18 18:44:28.272335 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:28.272334    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15572h59m18.732889746s"
Sep 18 18:44:28.307224 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:28.307154    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:28.307224 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:28.307172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:28.307411 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:28.307240    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:28.307411 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:28.307254    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:28.307411 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:28.307321    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:28.307411 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:28.307400    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:29.365761 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:29.365618    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal" event={"ID":"c29e46335904b8ccc50d8f973acddf9a","Type":"ContainerStarted","Data":"8c832183d72c78df663134c71111e397652e2244741b40fb5cbc84cbee914e5b"}
Sep 18 18:44:29.383419 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:29.382538    2581 generic.go:356] "Generic (PLEG): container finished" podID="09bfe3eb0f63920fa5521f093ea0db2b" containerID="8361a6b4ce59f88f7f4d6a21a20b839aaa0c1e292d34ba4acd626b326e2324ff" exitCode=0
Sep 18 18:44:29.383419 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:29.382587    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal" event={"ID":"09bfe3eb0f63920fa5521f093ea0db2b","Type":"ContainerDied","Data":"8361a6b4ce59f88f7f4d6a21a20b839aaa0c1e292d34ba4acd626b326e2324ff"}
Sep 18 18:44:29.400720 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:29.400638    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-52.ec2.internal" podStartSLOduration=4.400614892 podStartE2EDuration="4.400614892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:29.38326795 +0000 UTC m=+5.602275528" watchObservedRunningTime="2026-09-18 18:44:29.400614892 +0000 UTC m=+5.619622471"
Sep 18 18:44:29.847945 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:29.847859    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:29.847945 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:29.847925    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:29.848178 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:29.848055    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:29.848178 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:29.848115    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret podName:1dcfcf8b-4bd5-4919-9315-26fe6852ba7d nodeName:}" failed. No retries permitted until 2026-09-18 18:44:33.848097594 +0000 UTC m=+10.067105152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret") pod "global-pull-secret-syncer-zsrbq" (UID: "1dcfcf8b-4bd5-4919-9315-26fe6852ba7d") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:29.848178 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:29.848175    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:29.848302 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:29.848205    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs podName:85ea9986-0a9c-4b8a-835b-f42a46dd8e55 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:33.848195061 +0000 UTC m=+10.067202622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs") pod "network-metrics-daemon-skwt5" (UID: "85ea9986-0a9c-4b8a-835b-f42a46dd8e55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:30.050860 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:30.050188    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skwst\" (UniqueName: \"kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst\") pod \"network-check-target-9hxwx\" (UID: \"9a781944-d73f-4c37-8c50-8c71aaa9b2a6\") " pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:30.050860 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:30.050373    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:30.050860 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:30.050410    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:30.050860 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:30.050422    2581 projected.go:196] Error preparing data for projected volume kube-api-access-skwst for pod openshift-network-diagnostics/network-check-target-9hxwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:30.050860 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:30.050481    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst podName:9a781944-d73f-4c37-8c50-8c71aaa9b2a6 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:34.050462151 +0000 UTC m=+10.269469726 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-skwst" (UniqueName: "kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst") pod "network-check-target-9hxwx" (UID: "9a781944-d73f-4c37-8c50-8c71aaa9b2a6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:30.307977 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:30.307485    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:30.307977 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:30.307485    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:30.307977 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:30.307607    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:30.307977 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:30.307612    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:30.307977 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:30.307737    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:30.307977 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:30.307818    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:32.309145 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:32.309110    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:32.309589 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:32.309221    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:32.309666 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:32.309594    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:32.309736 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:32.309675    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:32.309876 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:32.309750    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:32.309934 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:32.309871    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:33.883381 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:33.883340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:33.883837 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:33.883403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:33.883837 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:33.883548    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:33.883837 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:33.883611    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret podName:1dcfcf8b-4bd5-4919-9315-26fe6852ba7d nodeName:}" failed. No retries permitted until 2026-09-18 18:44:41.883593926 +0000 UTC m=+18.102601498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret") pod "global-pull-secret-syncer-zsrbq" (UID: "1dcfcf8b-4bd5-4919-9315-26fe6852ba7d") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:33.884048 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:33.884029    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:33.884116 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:33.884089    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs podName:85ea9986-0a9c-4b8a-835b-f42a46dd8e55 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:41.884071885 +0000 UTC m=+18.103079461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs") pod "network-metrics-daemon-skwt5" (UID: "85ea9986-0a9c-4b8a-835b-f42a46dd8e55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:34.084928 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:34.084838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skwst\" (UniqueName: \"kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst\") pod \"network-check-target-9hxwx\" (UID: \"9a781944-d73f-4c37-8c50-8c71aaa9b2a6\") " pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:34.085077 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:34.085036    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:34.085077 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:34.085064    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:34.085178 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:34.085082    2581 projected.go:196] Error preparing data for projected volume kube-api-access-skwst for pod openshift-network-diagnostics/network-check-target-9hxwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:34.085178 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:34.085145    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst podName:9a781944-d73f-4c37-8c50-8c71aaa9b2a6 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:42.085126967 +0000 UTC m=+18.304134526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-skwst" (UniqueName: "kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst") pod "network-check-target-9hxwx" (UID: "9a781944-d73f-4c37-8c50-8c71aaa9b2a6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:34.312127 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:34.311915    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:34.312127 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:34.312058    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:34.312127 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:34.312067    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:34.312457 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:34.312177    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:34.312457 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:34.312062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:34.312457 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:34.312264    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:36.307260 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:36.306675    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:36.307260 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:36.306749    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:36.307260 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:36.306841    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:36.307260 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:36.306852    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:36.307260 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:36.306924    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:36.307260 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:36.307030    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:38.306667 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:38.306630    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:38.307332 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:38.306635    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:38.307332 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:38.306776    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:38.307332 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:38.306857    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:38.307332 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:38.306635    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:38.307332 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:38.306958    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:40.306811 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:40.306768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:40.307311 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:40.306780    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:40.307311 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:40.306925    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:40.307311 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:40.306982    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:40.307311 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:40.307093    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:40.307311 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:40.307188    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:41.941002 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:41.940802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:41.941541 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:41.941034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:41.941541 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:41.940975    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:41.941541 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:41.941136    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs podName:85ea9986-0a9c-4b8a-835b-f42a46dd8e55 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:57.941117205 +0000 UTC m=+34.160124764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs") pod "network-metrics-daemon-skwt5" (UID: "85ea9986-0a9c-4b8a-835b-f42a46dd8e55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:41.941541 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:41.941155    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:41.941541 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:41.941208    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret podName:1dcfcf8b-4bd5-4919-9315-26fe6852ba7d nodeName:}" failed. No retries permitted until 2026-09-18 18:44:57.941192135 +0000 UTC m=+34.160199712 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret") pod "global-pull-secret-syncer-zsrbq" (UID: "1dcfcf8b-4bd5-4919-9315-26fe6852ba7d") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:42.142760 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:42.142722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skwst\" (UniqueName: \"kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst\") pod \"network-check-target-9hxwx\" (UID: \"9a781944-d73f-4c37-8c50-8c71aaa9b2a6\") " pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:42.143012 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:42.142988    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:42.143055 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:42.143019    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:42.143055 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:42.143034    2581 projected.go:196] Error preparing data for projected volume kube-api-access-skwst for pod openshift-network-diagnostics/network-check-target-9hxwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:42.143129 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:42.143103    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst podName:9a781944-d73f-4c37-8c50-8c71aaa9b2a6 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:58.143082359 +0000 UTC m=+34.362089927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-skwst" (UniqueName: "kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst") pod "network-check-target-9hxwx" (UID: "9a781944-d73f-4c37-8c50-8c71aaa9b2a6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:42.307346 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:42.307266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:42.307488 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:42.307266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:42.307488 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:42.307395    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:42.307603 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:42.307478    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:42.307603 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:42.307280    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:42.307724 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:42.307602    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:44.307603 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:44.307570    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:44.308035 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:44.307662    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:44.308035 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:44.307759    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:44.308035 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:44.307890    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:44.308035 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:44.307934    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:44.308035 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:44.308006    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:45.422182 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.422147    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd7d39c2-89db-45d7-81b7-9ff9c7290424" containerID="3cc4aa7ed192881088fd09627932328f99b4ab215a6816021d4b1cd289b4d9c3" exitCode=0
Sep 18 18:44:45.423003 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.422253    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-twwkm" event={"ID":"dd7d39c2-89db-45d7-81b7-9ff9c7290424","Type":"ContainerDied","Data":"3cc4aa7ed192881088fd09627932328f99b4ab215a6816021d4b1cd289b4d9c3"}
Sep 18 18:44:45.425848 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.425815    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s" event={"ID":"fb0643b9-b435-4bc5-be37-8be315df6412","Type":"ContainerStarted","Data":"bdb4c16af6e957609120eef59463c029090475f1409a8a8a036a186eea8ea741"}
Sep 18 18:44:45.425971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.425892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s" event={"ID":"fb0643b9-b435-4bc5-be37-8be315df6412","Type":"ContainerStarted","Data":"673b9845143987637edc26d5711bf91938e67a84c4f252dcf58b1a89c969df20"}
Sep 18 18:44:45.425971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.425930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s" event={"ID":"fb0643b9-b435-4bc5-be37-8be315df6412","Type":"ContainerStarted","Data":"da75fc6f83b6808f755db9e181236ba406eb90f37da51f979213348a6c3a6f8d"}
Sep 18 18:44:45.425971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.425962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s" event={"ID":"fb0643b9-b435-4bc5-be37-8be315df6412","Type":"ContainerStarted","Data":"416ecd1d262746f004f1d95350b849803a5f829faf02f12561a28cdcad59a403"}
Sep 18 18:44:45.426092 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.426000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s" event={"ID":"fb0643b9-b435-4bc5-be37-8be315df6412","Type":"ContainerStarted","Data":"041dbbbcb120d71e6344342321ed4a2fc5897b704a4595e2bff7c63e8688dd72"}
Sep 18 18:44:45.426092 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.426028    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s" event={"ID":"fb0643b9-b435-4bc5-be37-8be315df6412","Type":"ContainerStarted","Data":"9ba8be89989c16b209a188a8a315a6160e6a61dd343d7cce8e043717a8343e28"}
Sep 18 18:44:45.427405 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.427378    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bspq5" event={"ID":"d2f53f4b-0b86-4f9a-a5f6-0c428539290f","Type":"ContainerStarted","Data":"8e825ad969c16ea3546a9f434d42b1aadb42ecff7455fcc56ae8004e8a1f05c2"}
Sep 18 18:44:45.430982 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.430936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j5f5r" event={"ID":"687f69e1-ac76-4d49-9a4d-c829f3d24f4e","Type":"ContainerStarted","Data":"c3e56c2b56f6e7e1d4ff5b4ed9a195b90d8bc51d593390ec96724f3e513646a7"}
Sep 18 18:44:45.433001 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.432930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57" event={"ID":"2415ad06-8ee8-4004-a147-65f2fe315aee","Type":"ContainerStarted","Data":"07d1f846b88d25b085ba85f9e2596843143235c9763fb82e30366d35e4c61af3"}
Sep 18 18:44:45.434266 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.434230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bxzsl" event={"ID":"cab91e29-3ede-43e0-9209-0945d73869a7","Type":"ContainerStarted","Data":"7a378dbcf65ca97b73e3666a926f2685bf1ade902f8aa2e4d0eba4abf2e86db6"}
Sep 18 18:44:45.436305 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.436279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal" event={"ID":"09bfe3eb0f63920fa5521f093ea0db2b","Type":"ContainerStarted","Data":"9ced5f515b7f3a2f1c22ffd3f9c4c2160145ab694da9e7a96cecac8e7df63578"}
Sep 18 18:44:45.437905 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.437879    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rqb8l" event={"ID":"aa308550-2945-40b0-ba91-87a9c8541846","Type":"ContainerStarted","Data":"7a8d5b5be3a93aa37215d401a48a714ebcfb297d41ce2fbf37bcf9deb9c2c4bc"}
Sep 18 18:44:45.439494 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.439470    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j2n6t" event={"ID":"6d1247b6-0d71-4c51-93b2-0f6407e179d6","Type":"ContainerStarted","Data":"381700e1790207615426b13bf5caae8db8f89e65de1abdaa425f40ca00eb5950"}
Sep 18 18:44:45.468150 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.468075    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j2n6t" podStartSLOduration=3.14591786 podStartE2EDuration="21.468060822s" totalImagesPullingTime="18.322142962s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:45.467825979 +0000 UTC m=+21.686833570" watchObservedRunningTime="2026-09-18 18:44:45.468060822 +0000 UTC m=+21.687068399"
Sep 18 18:44:45.485895 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.485834    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bspq5" podStartSLOduration=3.570956082 podStartE2EDuration="21.485815329s" totalImagesPullingTime="17.914859247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:45.485354658 +0000 UTC m=+21.704362235" watchObservedRunningTime="2026-09-18 18:44:45.485815329 +0000 UTC m=+21.704822908"
Sep 18 18:44:45.511838 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.511782    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rqb8l" podStartSLOduration=7.4476275 podStartE2EDuration="21.511765487s" totalImagesPullingTime="14.064137987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:45.511601793 +0000 UTC m=+21.730609370" watchObservedRunningTime="2026-09-18 18:44:45.511765487 +0000 UTC m=+21.730773063"
Sep 18 18:44:45.533554 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.533504    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-52.ec2.internal" podStartSLOduration=20.533489098 podStartE2EDuration="20.533489098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:45.533211561 +0000 UTC m=+21.752219137" watchObservedRunningTime="2026-09-18 18:44:45.533489098 +0000 UTC m=+21.752496674"
Sep 18 18:44:45.588787 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.588735    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bxzsl" podStartSLOduration=7.477026079 podStartE2EDuration="21.588695892s" totalImagesPullingTime="14.111669813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:45.559037391 +0000 UTC m=+21.778044967" watchObservedRunningTime="2026-09-18 18:44:45.588695892 +0000 UTC m=+21.807703545"
Sep 18 18:44:45.935456 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.935420    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:45.936147 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.936128    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:45.953487 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:45.953427    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-j5f5r" podStartSLOduration=3.469661324 podStartE2EDuration="21.953410118s" totalImagesPullingTime="18.483748794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:45.58904732 +0000 UTC m=+21.808054896" watchObservedRunningTime="2026-09-18 18:44:45.953410118 +0000 UTC m=+22.172417695"
Sep 18 18:44:46.307586 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:46.307557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:46.307749 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:46.307721    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:46.307997 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:46.307980    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:46.308091 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:46.308073    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:46.308128 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:46.308094    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:46.308218 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:46.308198    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:46.341956 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:46.341914    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 18:44:46.445247 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:46.445048    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57" event={"ID":"2415ad06-8ee8-4004-a147-65f2fe315aee","Type":"ContainerStarted","Data":"58a730aa44096ca42c9a3a59b0b4414ebace7a7c108535862b1b82c51cc744d4"}
Sep 18 18:44:47.274943 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:47.274805    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T18:44:46.341937291Z","UUID":"9e1a5fbf-a190-4fdd-bf2e-af5243288766","Handler":null,"Name":"","Endpoint":""}
Sep 18 18:44:47.277126 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:47.277085    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 18:44:47.277126 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:47.277127    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 18:44:47.448728 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:47.448648    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 18 18:44:47.449070 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:47.448640    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-469b7" event={"ID":"3cc0af90-41b6-4637-b38f-ee10ffc79837","Type":"ContainerStarted","Data":"86ceef0e01a3ff46eba05c3a4101883e037a9cbdb10a9102712d4d4467059729"}
Sep 18 18:44:48.307451 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:48.307413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:48.307652 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:48.307413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:48.307652 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:48.307548    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:48.307652 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:48.307413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:48.307652 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:48.307616    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:48.307855 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:48.307674    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:48.455005 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:48.454969    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s" event={"ID":"fb0643b9-b435-4bc5-be37-8be315df6412","Type":"ContainerStarted","Data":"ef765201c30daac4fb02361153afae1d9a5e7cd8e69fd1af0e82973428f8cdb7"}
Sep 18 18:44:48.457937 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:48.457420    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57" event={"ID":"2415ad06-8ee8-4004-a147-65f2fe315aee","Type":"ContainerStarted","Data":"617376254d534b982c5998b70efb5c806136a2c8cc6b6a972d9241cba225a781"}
Sep 18 18:44:48.476059 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:48.476004    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-469b7" podStartSLOduration=6.217357639 podStartE2EDuration="24.475985047s" totalImagesPullingTime="18.258627408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:47.480578784 +0000 UTC m=+23.699586360" watchObservedRunningTime="2026-09-18 18:44:48.475985047 +0000 UTC m=+24.694992637"
Sep 18 18:44:48.476260 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:48.476239    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l57" podStartSLOduration=3.955022332 podStartE2EDuration="24.476233238s" totalImagesPullingTime="20.521210906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:48.475767455 +0000 UTC m=+24.694775033" watchObservedRunningTime="2026-09-18 18:44:48.476233238 +0000 UTC m=+24.695240815"
Sep 18 18:44:50.307068 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:50.307025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:50.307579 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:50.307070    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:50.307579 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:50.307027    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:50.307579 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:50.307156    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:50.307579 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:50.307285    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:50.307579 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:50.307375    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:51.244820 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:51.244599    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:51.244973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:51.244937    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 18 18:44:51.245332 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:51.245308    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bxzsl"
Sep 18 18:44:51.466168 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:51.466125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-twwkm" event={"ID":"dd7d39c2-89db-45d7-81b7-9ff9c7290424","Type":"ContainerStarted","Data":"ff23b1096d00572665c74b3a73253a6aae6d6141e1dac6f8b58e2299859cfd92"}
Sep 18 18:44:51.470156 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:51.470122    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s" event={"ID":"fb0643b9-b435-4bc5-be37-8be315df6412","Type":"ContainerStarted","Data":"6f18d8ed01fe4f3fbb327d7a5a16b0876ee0959a27c2b327b0db6ff3565c1fc8"}
Sep 18 18:44:52.307454 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.307415    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:52.307692 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.307415    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:52.307692 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:52.307555    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:52.307692 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.307412    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:52.307692 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:52.307599    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:52.307879 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:52.307689    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:52.474870 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.474831    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd7d39c2-89db-45d7-81b7-9ff9c7290424" containerID="ff23b1096d00572665c74b3a73253a6aae6d6141e1dac6f8b58e2299859cfd92" exitCode=0
Sep 18 18:44:52.475380 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.474914    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-twwkm" event={"ID":"dd7d39c2-89db-45d7-81b7-9ff9c7290424","Type":"ContainerDied","Data":"ff23b1096d00572665c74b3a73253a6aae6d6141e1dac6f8b58e2299859cfd92"}
Sep 18 18:44:52.475442 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.475429    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:52.475493 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.475486    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:52.475540 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.475535    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:52.493174 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.493146    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:52.493613 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.493597    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:44:52.508660 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:52.508614    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s" podStartSLOduration=10.21471301 podStartE2EDuration="28.508596932s" totalImagesPullingTime="18.293883922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:51.545997227 +0000 UTC m=+27.765004805" watchObservedRunningTime="2026-09-18 18:44:52.508596932 +0000 UTC m=+28.727604509"
Sep 18 18:44:53.077556 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:53.077464    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zsrbq"]
Sep 18 18:44:53.077740 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:53.077588    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:53.077740 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:53.077692    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:53.082214 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:53.082160    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-skwt5"]
Sep 18 18:44:53.082334 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:53.082318    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:53.082437 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:53.082419    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:53.084787 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:53.084763    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9hxwx"]
Sep 18 18:44:53.084877 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:53.084858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:53.084973 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:53.084950    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:53.478994 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:53.478962    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd7d39c2-89db-45d7-81b7-9ff9c7290424" containerID="edb1b270289086882c65955f51394eaa64d63d082ac4a51e0f7d67f52fe9ae91" exitCode=0
Sep 18 18:44:53.479656 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:53.479047    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-twwkm" event={"ID":"dd7d39c2-89db-45d7-81b7-9ff9c7290424","Type":"ContainerDied","Data":"edb1b270289086882c65955f51394eaa64d63d082ac4a51e0f7d67f52fe9ae91"}
Sep 18 18:44:54.307016 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:54.306489    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:54.308302 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:54.308255    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:54.483534 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:54.483424    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd7d39c2-89db-45d7-81b7-9ff9c7290424" containerID="96d78a5e19a83f11dd879273854cf325474a26e22e5d90e26ffd5cc85e050cca" exitCode=0
Sep 18 18:44:54.483534 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:54.483490    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-twwkm" event={"ID":"dd7d39c2-89db-45d7-81b7-9ff9c7290424","Type":"ContainerDied","Data":"96d78a5e19a83f11dd879273854cf325474a26e22e5d90e26ffd5cc85e050cca"}
Sep 18 18:44:55.306527 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:55.306484    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:55.306725 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:55.306486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:55.306725 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:55.306609    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:55.306850 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:55.306717    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:56.306746 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:56.306688    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:56.307180 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:56.306849    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hxwx" podUID="9a781944-d73f-4c37-8c50-8c71aaa9b2a6"
Sep 18 18:44:57.307243 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.307207    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:57.307695 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.307348    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsrbq" podUID="1dcfcf8b-4bd5-4919-9315-26fe6852ba7d"
Sep 18 18:44:57.307695 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.307391    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:57.307695 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.307613    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skwt5" podUID="85ea9986-0a9c-4b8a-835b-f42a46dd8e55"
Sep 18 18:44:57.575619 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.575535    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-52.ec2.internal" event="NodeReady"
Sep 18 18:44:57.575816 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.575721    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 18:44:57.654342 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.654296    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64c8d855c6-zrzwq"]
Sep 18 18:44:57.682564 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.682518    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-smkgk"]
Sep 18 18:44:57.682748 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.682660    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.685606 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.685573    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 18:44:57.685800 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.685687    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m6jx4\""
Sep 18 18:44:57.686119 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.685573    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 18:44:57.687358 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.687334    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 18:44:57.693328 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.693260    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 18:44:57.700520 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.700488    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64c8d855c6-zrzwq"]
Sep 18 18:44:57.700690 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.700531    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z4rg2"]
Sep 18 18:44:57.700690 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.700645    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.703340 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.703313    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 18:44:57.703340 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.703330    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 18:44:57.703512 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.703416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qf4nw\""
Sep 18 18:44:57.718971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.718924    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-smkgk"]
Sep 18 18:44:57.718971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.718981    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z4rg2"]
Sep 18 18:44:57.719175 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.719083    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z4rg2"
Sep 18 18:44:57.722325 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.722299    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 18:44:57.722795 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.722572    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 18:44:57.722903 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.722842    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 18:44:57.723630 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.723608    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sl7vh\""
Sep 18 18:44:57.762426 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762386    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-installation-pull-secrets\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.762426 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762435    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-registry-certificates\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.762676 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762520    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.762676 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-bound-sa-token\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.762676 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/946a2586-1164-4bdf-9348-22b7089112e0-config-volume\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.762676 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-image-registry-private-configuration\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.762676 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762636    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rksv\" (UniqueName: \"kubernetes.io/projected/946a2586-1164-4bdf-9348-22b7089112e0-kube-api-access-7rksv\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.762676 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762673    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk996\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-kube-api-access-tk996\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.763021 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.763021 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6273c63-ae77-482f-bee7-7399f9793173-ca-trust-extracted\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.763021 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-trusted-ca\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.763021 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.762805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946a2586-1164-4bdf-9348-22b7089112e0-tmp-dir\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.845869 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.845761    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bxzsl_cab91e29-3ede-43e0-9209-0945d73869a7/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:57.863788 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.863751    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk996\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-kube-api-access-tk996\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.863971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.863801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.863971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.863835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b51682e9-4379-4076-a853-ec0306df5d58-cert\") pod \"ingress-canary-z4rg2\" (UID: \"b51682e9-4379-4076-a853-ec0306df5d58\") " pod="openshift-ingress-canary/ingress-canary-z4rg2"
Sep 18 18:44:57.863971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.863866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6273c63-ae77-482f-bee7-7399f9793173-ca-trust-extracted\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.863971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.863900    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-trusted-ca\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.863971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.863927    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946a2586-1164-4bdf-9348-22b7089112e0-tmp-dir\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.863971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.863960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-installation-pull-secrets\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.864252 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.864001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-registry-certificates\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.864252 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.864040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.864252 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.864068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-bound-sa-token\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.864252 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.864084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/946a2586-1164-4bdf-9348-22b7089112e0-config-volume\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.864252 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.864107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msbzt\" (UniqueName: \"kubernetes.io/projected/b51682e9-4379-4076-a853-ec0306df5d58-kube-api-access-msbzt\") pod \"ingress-canary-z4rg2\" (UID: \"b51682e9-4379-4076-a853-ec0306df5d58\") " pod="openshift-ingress-canary/ingress-canary-z4rg2"
Sep 18 18:44:57.864252 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.864133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-image-registry-private-configuration\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.864252 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.864164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rksv\" (UniqueName: \"kubernetes.io/projected/946a2586-1164-4bdf-9348-22b7089112e0-kube-api-access-7rksv\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.864568 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.864419    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946a2586-1164-4bdf-9348-22b7089112e0-tmp-dir\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.864568 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.864522    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:44:57.864568 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.864537    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64c8d855c6-zrzwq: secret "image-registry-tls" not found
Sep 18 18:44:57.864735 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.864584    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls podName:e6273c63-ae77-482f-bee7-7399f9793173 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:58.364564801 +0000 UTC m=+34.583572357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls") pod "image-registry-64c8d855c6-zrzwq" (UID: "e6273c63-ae77-482f-bee7-7399f9793173") : secret "image-registry-tls" not found
Sep 18 18:44:57.865795 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.864921    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:44:57.865795 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.864986    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls podName:946a2586-1164-4bdf-9348-22b7089112e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:58.364972726 +0000 UTC m=+34.583980294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls") pod "dns-default-smkgk" (UID: "946a2586-1164-4bdf-9348-22b7089112e0") : secret "dns-default-metrics-tls" not found
Sep 18 18:44:57.865795 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.865306    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/946a2586-1164-4bdf-9348-22b7089112e0-config-volume\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.865795 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.865321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6273c63-ae77-482f-bee7-7399f9793173-ca-trust-extracted\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.865795 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.865726    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-registry-certificates\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.866595 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.866569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-trusted-ca\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.870090 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.870044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-installation-pull-secrets\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.870204 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.870106    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-image-registry-private-configuration\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.874554 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.874527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk996\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-kube-api-access-tk996\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.874747 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.874712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-bound-sa-token\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:57.875336 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.875274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rksv\" (UniqueName: \"kubernetes.io/projected/946a2586-1164-4bdf-9348-22b7089112e0-kube-api-access-7rksv\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:57.939605 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.939573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bxzsl_cab91e29-3ede-43e0-9209-0945d73869a7/konnectivity-agent/0.log" limit=1024
Sep 18 18:44:57.964678 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.964636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:57.964877 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.964737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msbzt\" (UniqueName: \"kubernetes.io/projected/b51682e9-4379-4076-a853-ec0306df5d58-kube-api-access-msbzt\") pod \"ingress-canary-z4rg2\" (UID: \"b51682e9-4379-4076-a853-ec0306df5d58\") " pod="openshift-ingress-canary/ingress-canary-z4rg2"
Sep 18 18:44:57.964877 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.964774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:57.964877 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.964823    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b51682e9-4379-4076-a853-ec0306df5d58-cert\") pod \"ingress-canary-z4rg2\" (UID: \"b51682e9-4379-4076-a853-ec0306df5d58\") " pod="openshift-ingress-canary/ingress-canary-z4rg2"
Sep 18 18:44:57.964877 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.964831    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:57.965083 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.964937    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs podName:85ea9986-0a9c-4b8a-835b-f42a46dd8e55 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:29.964917419 +0000 UTC m=+66.183924997 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs") pod "network-metrics-daemon-skwt5" (UID: "85ea9986-0a9c-4b8a-835b-f42a46dd8e55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:57.965083 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.964942    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:57.965083 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:57.965000    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret podName:1dcfcf8b-4bd5-4919-9315-26fe6852ba7d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:29.964982865 +0000 UTC m=+66.183990435 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret") pod "global-pull-secret-syncer-zsrbq" (UID: "1dcfcf8b-4bd5-4919-9315-26fe6852ba7d") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:57.967697 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.967662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b51682e9-4379-4076-a853-ec0306df5d58-cert\") pod \"ingress-canary-z4rg2\" (UID: \"b51682e9-4379-4076-a853-ec0306df5d58\") " pod="openshift-ingress-canary/ingress-canary-z4rg2"
Sep 18 18:44:57.973936 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:57.973912    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msbzt\" (UniqueName: \"kubernetes.io/projected/b51682e9-4379-4076-a853-ec0306df5d58-kube-api-access-msbzt\") pod \"ingress-canary-z4rg2\" (UID: \"b51682e9-4379-4076-a853-ec0306df5d58\") " pod="openshift-ingress-canary/ingress-canary-z4rg2"
Sep 18 18:44:58.030608 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:58.030567    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z4rg2"
Sep 18 18:44:58.166124 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:58.166087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skwst\" (UniqueName: \"kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst\") pod \"network-check-target-9hxwx\" (UID: \"9a781944-d73f-4c37-8c50-8c71aaa9b2a6\") " pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:58.166322 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:58.166257    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:58.166322 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:58.166277    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:58.166322 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:58.166287    2581 projected.go:196] Error preparing data for projected volume kube-api-access-skwst for pod openshift-network-diagnostics/network-check-target-9hxwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:58.166490 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:58.166348    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst podName:9a781944-d73f-4c37-8c50-8c71aaa9b2a6 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:30.166334764 +0000 UTC m=+66.385342320 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-skwst" (UniqueName: "kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst") pod "network-check-target-9hxwx" (UID: "9a781944-d73f-4c37-8c50-8c71aaa9b2a6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:58.307268 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:58.307229    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:44:58.310302 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:58.310276    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 18:44:58.310438 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:58.310332    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 18:44:58.310438 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:58.310382    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vk59r\""
Sep 18 18:44:58.368045 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:58.368004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:58.368229 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:58.368106    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:58.368229 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:58.368181    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:44:58.368229 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:58.368203    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64c8d855c6-zrzwq: secret "image-registry-tls" not found
Sep 18 18:44:58.368229 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:58.368222    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:44:58.368395 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:58.368286    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls podName:946a2586-1164-4bdf-9348-22b7089112e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:59.368267689 +0000 UTC m=+35.587275257 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls") pod "dns-default-smkgk" (UID: "946a2586-1164-4bdf-9348-22b7089112e0") : secret "dns-default-metrics-tls" not found
Sep 18 18:44:58.368395 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:58.368304    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls podName:e6273c63-ae77-482f-bee7-7399f9793173 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:59.368295159 +0000 UTC m=+35.587302720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls") pod "image-registry-64c8d855c6-zrzwq" (UID: "e6273c63-ae77-482f-bee7-7399f9793173") : secret "image-registry-tls" not found
Sep 18 18:44:59.306966 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:59.306919    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:44:59.307154 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:59.307107    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:44:59.309593 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:59.309548    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:44:59.310894 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:59.310872    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xdsv4\""
Sep 18 18:44:59.310983 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:59.310884    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:44:59.375498 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:59.375464    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:44:59.375657 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:59.375537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:44:59.375657 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:59.375617    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:44:59.375747 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:59.375672    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls podName:946a2586-1164-4bdf-9348-22b7089112e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:01.375657906 +0000 UTC m=+37.594665461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls") pod "dns-default-smkgk" (UID: "946a2586-1164-4bdf-9348-22b7089112e0") : secret "dns-default-metrics-tls" not found
Sep 18 18:44:59.375747 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:59.375621    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:44:59.375747 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:59.375723    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64c8d855c6-zrzwq: secret "image-registry-tls" not found
Sep 18 18:44:59.375842 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:44:59.375760    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls podName:e6273c63-ae77-482f-bee7-7399f9793173 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:01.375749418 +0000 UTC m=+37.594756974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls") pod "image-registry-64c8d855c6-zrzwq" (UID: "e6273c63-ae77-482f-bee7-7399f9793173") : secret "image-registry-tls" not found
Sep 18 18:44:59.411550 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:59.411332    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z4rg2"]
Sep 18 18:44:59.415462 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:44:59.415411    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb51682e9_4379_4076_a853_ec0306df5d58.slice/crio-dc804c305ef5058228cd3b879e09a50f38aebd01a474b517b0d019707cc0a699 WatchSource:0}: Error finding container dc804c305ef5058228cd3b879e09a50f38aebd01a474b517b0d019707cc0a699: Status 404 returned error can't find the container with id dc804c305ef5058228cd3b879e09a50f38aebd01a474b517b0d019707cc0a699
Sep 18 18:44:59.497323 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:59.497287    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z4rg2" event={"ID":"b51682e9-4379-4076-a853-ec0306df5d58","Type":"ContainerStarted","Data":"dc804c305ef5058228cd3b879e09a50f38aebd01a474b517b0d019707cc0a699"}
Sep 18 18:44:59.500294 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:44:59.500265    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-twwkm" event={"ID":"dd7d39c2-89db-45d7-81b7-9ff9c7290424","Type":"ContainerStarted","Data":"a12a74efa96f81c287ca47e95ea524cf8116b23b4e388529130c1ab9dbd76ed3"}
Sep 18 18:45:00.506618 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:00.506581    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd7d39c2-89db-45d7-81b7-9ff9c7290424" containerID="a12a74efa96f81c287ca47e95ea524cf8116b23b4e388529130c1ab9dbd76ed3" exitCode=0
Sep 18 18:45:00.507100 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:00.506649    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-twwkm" event={"ID":"dd7d39c2-89db-45d7-81b7-9ff9c7290424","Type":"ContainerDied","Data":"a12a74efa96f81c287ca47e95ea524cf8116b23b4e388529130c1ab9dbd76ed3"}
Sep 18 18:45:01.392072 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:01.392025    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:45:01.392274 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:01.392111    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:45:01.392274 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:01.392247    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:45:01.392274 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:01.392247    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:45:01.392434 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:01.392314    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls podName:946a2586-1164-4bdf-9348-22b7089112e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:05.392298547 +0000 UTC m=+41.611306106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls") pod "dns-default-smkgk" (UID: "946a2586-1164-4bdf-9348-22b7089112e0") : secret "dns-default-metrics-tls" not found
Sep 18 18:45:01.392724 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:01.392259    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64c8d855c6-zrzwq: secret "image-registry-tls" not found
Sep 18 18:45:01.392912 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:01.392752    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls podName:e6273c63-ae77-482f-bee7-7399f9793173 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:05.392738388 +0000 UTC m=+41.611745943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls") pod "image-registry-64c8d855c6-zrzwq" (UID: "e6273c63-ae77-482f-bee7-7399f9793173") : secret "image-registry-tls" not found
Sep 18 18:45:01.513560 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:01.513524    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd7d39c2-89db-45d7-81b7-9ff9c7290424" containerID="ac64a797ccd42d1c68d3b2e255e86b92725ecb7e414885320b78ff23d8c3831d" exitCode=0
Sep 18 18:45:01.514305 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:01.513572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-twwkm" event={"ID":"dd7d39c2-89db-45d7-81b7-9ff9c7290424","Type":"ContainerDied","Data":"ac64a797ccd42d1c68d3b2e255e86b92725ecb7e414885320b78ff23d8c3831d"}
Sep 18 18:45:02.517178 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:02.517138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z4rg2" event={"ID":"b51682e9-4379-4076-a853-ec0306df5d58","Type":"ContainerStarted","Data":"2455b1ffdc618eee9130a02b70768d02abea5294fefab02eb7785c0a6c9c05fb"}
Sep 18 18:45:02.520646 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:02.520617    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-twwkm" event={"ID":"dd7d39c2-89db-45d7-81b7-9ff9c7290424","Type":"ContainerStarted","Data":"8753ea265ab43c3cbd55de4390a57e5c24723a40109f8bf350ac6bc75bab27ca"}
Sep 18 18:45:02.533619 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:02.533565    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z4rg2" podStartSLOduration=2.86339259 podStartE2EDuration="5.533548038s" totalImagesPullingTime="2.670155448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:02.533166552 +0000 UTC m=+38.752174127" watchObservedRunningTime="2026-09-18 18:45:02.533548038 +0000 UTC m=+38.752555614"
Sep 18 18:45:02.558470 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:02.558419    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-twwkm" podStartSLOduration=7.856028253 podStartE2EDuration="38.558399233s" totalImagesPullingTime="30.70237098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:02.557025984 +0000 UTC m=+38.776033561" watchObservedRunningTime="2026-09-18 18:45:02.558399233 +0000 UTC m=+38.777406810"
Sep 18 18:45:05.422329 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:05.422282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:45:05.422831 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:05.422372    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:45:05.422831 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:05.422452    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:45:05.422831 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:05.422490    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:45:05.422831 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:05.422507    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64c8d855c6-zrzwq: secret "image-registry-tls" not found
Sep 18 18:45:05.422831 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:05.422524    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls podName:946a2586-1164-4bdf-9348-22b7089112e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:13.422507681 +0000 UTC m=+49.641515237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls") pod "dns-default-smkgk" (UID: "946a2586-1164-4bdf-9348-22b7089112e0") : secret "dns-default-metrics-tls" not found
Sep 18 18:45:05.422831 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:05.422556    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls podName:e6273c63-ae77-482f-bee7-7399f9793173 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:13.42254116 +0000 UTC m=+49.641548721 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls") pod "image-registry-64c8d855c6-zrzwq" (UID: "e6273c63-ae77-482f-bee7-7399f9793173") : secret "image-registry-tls" not found
Sep 18 18:45:09.821867 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.821828    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"]
Sep 18 18:45:09.851718 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.851681    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn"]
Sep 18 18:45:09.851933 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.851836    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:09.855195 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.855160    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 18:45:09.855374 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.855161    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 18:45:09.855374 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.855264    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 18 18:45:09.857196 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.857173    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nb7fx\""
Sep 18 18:45:09.858900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.858878    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 18 18:45:09.863379 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.863350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"]
Sep 18 18:45:09.863505 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.863388    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn"]
Sep 18 18:45:09.863505 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.863465    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn"
Sep 18 18:45:09.868445 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.867661    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:45:09.868445 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.867833    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 18 18:45:09.868445 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.867902    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7jq9j\""
Sep 18 18:45:09.935937 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.935904    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"]
Sep 18 18:45:09.957054 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.957021    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:09.957219 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.957174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgccg\" (UniqueName: \"kubernetes.io/projected/b14702cc-c97a-45ba-8fac-466e1e76d9fa-kube-api-access-tgccg\") pod \"volume-data-source-validator-5c4db8bdf9-k84bn\" (UID: \"b14702cc-c97a-45ba-8fac-466e1e76d9fa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn"
Sep 18 18:45:09.957291 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.957271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:09.957344 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.957310    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:09.957344 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.957335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2tq4\" (UniqueName: \"kubernetes.io/projected/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-kube-api-access-f2tq4\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:09.964504 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.964474    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 18 18:45:09.964662 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.964596    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:45:09.964662 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.964638    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 18 18:45:09.964946 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.964929    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 18 18:45:09.965047 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.965031    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2rqj6\""
Sep 18 18:45:09.965626 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:09.965607    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"]
Sep 18 18:45:10.043493 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.043458    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"]
Sep 18 18:45:10.058448 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.058407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:10.058448 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.058453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:10.058670 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.058472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2tq4\" (UniqueName: \"kubernetes.io/projected/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-kube-api-access-f2tq4\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:10.058670 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.058496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf71c2eb-68c0-4f63-94e8-d71050926938-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-x8vmt\" (UID: \"cf71c2eb-68c0-4f63-94e8-d71050926938\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:10.058670 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.058609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgccg\" (UniqueName: \"kubernetes.io/projected/b14702cc-c97a-45ba-8fac-466e1e76d9fa-kube-api-access-tgccg\") pod \"volume-data-source-validator-5c4db8bdf9-k84bn\" (UID: \"b14702cc-c97a-45ba-8fac-466e1e76d9fa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn"
Sep 18 18:45:10.058670 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.058646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf71c2eb-68c0-4f63-94e8-d71050926938-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-x8vmt\" (UID: \"cf71c2eb-68c0-4f63-94e8-d71050926938\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:10.058964 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.058674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x4lk\" (UniqueName: \"kubernetes.io/projected/cf71c2eb-68c0-4f63-94e8-d71050926938-kube-api-access-9x4lk\") pod \"kube-storage-version-migrator-operator-55889f7b47-x8vmt\" (UID: \"cf71c2eb-68c0-4f63-94e8-d71050926938\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:10.058964 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.058725    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:10.058964 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.058810    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls podName:a7fa0c11-eafd-4961-a6ab-2d03d0e571db nodeName:}" failed. No retries permitted until 2026-09-18 18:45:10.5587896 +0000 UTC m=+46.777797170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-q8mj9" (UID: "a7fa0c11-eafd-4961-a6ab-2d03d0e571db") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:10.059209 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.059190    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:10.059258 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.059242    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"]
Sep 18 18:45:10.059359 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.059346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:10.066579 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.066553    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 18 18:45:10.067175 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.067152    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 18 18:45:10.067586 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.067568    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jw2lz\""
Sep 18 18:45:10.078319 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.078286    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2tq4\" (UniqueName: \"kubernetes.io/projected/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-kube-api-access-f2tq4\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:10.078319 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.078306    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgccg\" (UniqueName: \"kubernetes.io/projected/b14702cc-c97a-45ba-8fac-466e1e76d9fa-kube-api-access-tgccg\") pod \"volume-data-source-validator-5c4db8bdf9-k84bn\" (UID: \"b14702cc-c97a-45ba-8fac-466e1e76d9fa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn"
Sep 18 18:45:10.080955 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.080916    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"]
Sep 18 18:45:10.080955 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.080952    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"]
Sep 18 18:45:10.081145 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.081055    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:10.086921 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.086894    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 18 18:45:10.087093 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.086894    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 18 18:45:10.087376 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.087361    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:45:10.087724 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.087691    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-n5tz6\""
Sep 18 18:45:10.159299 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.159265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpv6d\" (UniqueName: \"kubernetes.io/projected/7a9675cf-d635-42ea-b0f6-dd3c65541273-kube-api-access-kpv6d\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:10.159484 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.159308    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf71c2eb-68c0-4f63-94e8-d71050926938-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-x8vmt\" (UID: \"cf71c2eb-68c0-4f63-94e8-d71050926938\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:10.159484 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.159329    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c9085e34-f699-4aec-9aa7-15b7a35766d7-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:10.159484 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.159351    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x4lk\" (UniqueName: \"kubernetes.io/projected/cf71c2eb-68c0-4f63-94e8-d71050926938-kube-api-access-9x4lk\") pod \"kube-storage-version-migrator-operator-55889f7b47-x8vmt\" (UID: \"cf71c2eb-68c0-4f63-94e8-d71050926938\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:10.159599 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.159560    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:10.159640 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.159606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf71c2eb-68c0-4f63-94e8-d71050926938-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-x8vmt\" (UID: \"cf71c2eb-68c0-4f63-94e8-d71050926938\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:10.159640 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.159631    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:10.159923 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.159903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf71c2eb-68c0-4f63-94e8-d71050926938-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-x8vmt\" (UID: \"cf71c2eb-68c0-4f63-94e8-d71050926938\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:10.162025 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.162003    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf71c2eb-68c0-4f63-94e8-d71050926938-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-x8vmt\" (UID: \"cf71c2eb-68c0-4f63-94e8-d71050926938\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:10.173754 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.173717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn"
Sep 18 18:45:10.175235 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.175204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x4lk\" (UniqueName: \"kubernetes.io/projected/cf71c2eb-68c0-4f63-94e8-d71050926938-kube-api-access-9x4lk\") pod \"kube-storage-version-migrator-operator-55889f7b47-x8vmt\" (UID: \"cf71c2eb-68c0-4f63-94e8-d71050926938\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:10.260304 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.260233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:10.260304 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.260289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:10.260545 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.260370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpv6d\" (UniqueName: \"kubernetes.io/projected/7a9675cf-d635-42ea-b0f6-dd3c65541273-kube-api-access-kpv6d\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:10.260545 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.260378    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:45:10.260545 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.260411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c9085e34-f699-4aec-9aa7-15b7a35766d7-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:10.260545 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.260451    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:45:10.260545 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.260465    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls podName:7a9675cf-d635-42ea-b0f6-dd3c65541273 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:10.76044449 +0000 UTC m=+46.979452061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-875v9" (UID: "7a9675cf-d635-42ea-b0f6-dd3c65541273") : secret "samples-operator-tls" not found
Sep 18 18:45:10.260545 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.260516    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert podName:c9085e34-f699-4aec-9aa7-15b7a35766d7 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:10.760497194 +0000 UTC m=+46.979504764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-7r7xw" (UID: "c9085e34-f699-4aec-9aa7-15b7a35766d7") : secret "networking-console-plugin-cert" not found
Sep 18 18:45:10.261174 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.261153    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c9085e34-f699-4aec-9aa7-15b7a35766d7-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:10.267167 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.267136    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"
Sep 18 18:45:10.277005 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.276972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpv6d\" (UniqueName: \"kubernetes.io/projected/7a9675cf-d635-42ea-b0f6-dd3c65541273-kube-api-access-kpv6d\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:10.481407 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.481315    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn"]
Sep 18 18:45:10.488955 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:45:10.488914    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14702cc_c97a_45ba_8fac_466e1e76d9fa.slice/crio-fc5064c26a5cc07a7853ecaac0f6b4c6f32ab8998b2bd68733d80ffe61d49952 WatchSource:0}: Error finding container fc5064c26a5cc07a7853ecaac0f6b4c6f32ab8998b2bd68733d80ffe61d49952: Status 404 returned error can't find the container with id fc5064c26a5cc07a7853ecaac0f6b4c6f32ab8998b2bd68733d80ffe61d49952
Sep 18 18:45:10.540424 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.540384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn" event={"ID":"b14702cc-c97a-45ba-8fac-466e1e76d9fa","Type":"ContainerStarted","Data":"fc5064c26a5cc07a7853ecaac0f6b4c6f32ab8998b2bd68733d80ffe61d49952"}
Sep 18 18:45:10.562871 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.562833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:10.563055 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.562991    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:10.563100 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.563062    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls podName:a7fa0c11-eafd-4961-a6ab-2d03d0e571db nodeName:}" failed. No retries permitted until 2026-09-18 18:45:11.563047119 +0000 UTC m=+47.782054675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-q8mj9" (UID: "a7fa0c11-eafd-4961-a6ab-2d03d0e571db") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:10.586740 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.586686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt"]
Sep 18 18:45:10.589849 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:45:10.589812    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf71c2eb_68c0_4f63_94e8_d71050926938.slice/crio-8c7d35543d57abfc01ef919c1d147101cf97b396b3fa4237b68697f00b2144b7 WatchSource:0}: Error finding container 8c7d35543d57abfc01ef919c1d147101cf97b396b3fa4237b68697f00b2144b7: Status 404 returned error can't find the container with id 8c7d35543d57abfc01ef919c1d147101cf97b396b3fa4237b68697f00b2144b7
Sep 18 18:45:10.765067 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.764962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:10.765067 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:10.765008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:10.765258 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.765101    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:45:10.765258 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.765127    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:45:10.765258 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.765157    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert podName:c9085e34-f699-4aec-9aa7-15b7a35766d7 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:11.765142662 +0000 UTC m=+47.984150218 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-7r7xw" (UID: "c9085e34-f699-4aec-9aa7-15b7a35766d7") : secret "networking-console-plugin-cert" not found
Sep 18 18:45:10.765258 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:10.765189    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls podName:7a9675cf-d635-42ea-b0f6-dd3c65541273 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:11.76517523 +0000 UTC m=+47.984182786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-875v9" (UID: "7a9675cf-d635-42ea-b0f6-dd3c65541273") : secret "samples-operator-tls" not found
Sep 18 18:45:11.544633 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:11.544578    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt" event={"ID":"cf71c2eb-68c0-4f63-94e8-d71050926938","Type":"ContainerStarted","Data":"8c7d35543d57abfc01ef919c1d147101cf97b396b3fa4237b68697f00b2144b7"}
Sep 18 18:45:11.572225 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:11.572189    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:11.572408 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:11.572368    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:11.572491 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:11.572452    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls podName:a7fa0c11-eafd-4961-a6ab-2d03d0e571db nodeName:}" failed. No retries permitted until 2026-09-18 18:45:13.572430735 +0000 UTC m=+49.791438293 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-q8mj9" (UID: "a7fa0c11-eafd-4961-a6ab-2d03d0e571db") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:11.774117 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:11.774072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:11.774315 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:11.774133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:11.774315 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:11.774250    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:45:11.774432 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:11.774332    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls podName:7a9675cf-d635-42ea-b0f6-dd3c65541273 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:13.774309774 +0000 UTC m=+49.993317345 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-875v9" (UID: "7a9675cf-d635-42ea-b0f6-dd3c65541273") : secret "samples-operator-tls" not found
Sep 18 18:45:11.774432 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:11.774344    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:45:11.774432 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:11.774386    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert podName:c9085e34-f699-4aec-9aa7-15b7a35766d7 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:13.774374424 +0000 UTC m=+49.993381980 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-7r7xw" (UID: "c9085e34-f699-4aec-9aa7-15b7a35766d7") : secret "networking-console-plugin-cert" not found
Sep 18 18:45:13.489190 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:13.489150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:45:13.489590 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:13.489249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:45:13.489590 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.489338    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:45:13.489590 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.489342    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:45:13.489590 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.489421    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls podName:946a2586-1164-4bdf-9348-22b7089112e0 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:29.489401181 +0000 UTC m=+65.708408737 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls") pod "dns-default-smkgk" (UID: "946a2586-1164-4bdf-9348-22b7089112e0") : secret "dns-default-metrics-tls" not found
Sep 18 18:45:13.489590 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.489350    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64c8d855c6-zrzwq: secret "image-registry-tls" not found
Sep 18 18:45:13.489590 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.489477    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls podName:e6273c63-ae77-482f-bee7-7399f9793173 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:29.489466022 +0000 UTC m=+65.708473578 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls") pod "image-registry-64c8d855c6-zrzwq" (UID: "e6273c63-ae77-482f-bee7-7399f9793173") : secret "image-registry-tls" not found
Sep 18 18:45:13.552255 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:13.552158    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn" event={"ID":"b14702cc-c97a-45ba-8fac-466e1e76d9fa","Type":"ContainerStarted","Data":"e48648071c553f735e9fd35448e559d1a5c331333c2cd41989a52a210362c096"}
Sep 18 18:45:13.553831 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:13.553793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt" event={"ID":"cf71c2eb-68c0-4f63-94e8-d71050926938","Type":"ContainerStarted","Data":"8ae5be1cb6607ce18b9429950a3339612eed052953f00e839b27345f4dee71c1"}
Sep 18 18:45:13.571226 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:13.571165    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-k84bn" podStartSLOduration=2.058362787 podStartE2EDuration="4.571148771s" totalImagesPullingTime="2.512785984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:13.570395908 +0000 UTC m=+49.789403486" watchObservedRunningTime="2026-09-18 18:45:13.571148771 +0000 UTC m=+49.790156351"
Sep 18 18:45:13.587994 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:13.587939    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt" podStartSLOduration=1.8864219100000001 podStartE2EDuration="4.58791964s" totalImagesPullingTime="2.70149773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:13.587491869 +0000 UTC m=+49.806499452" watchObservedRunningTime="2026-09-18 18:45:13.58791964 +0000 UTC m=+49.806927217"
Sep 18 18:45:13.589867 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:13.589838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:13.590012 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.589996    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:13.590070 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.590058    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls podName:a7fa0c11-eafd-4961-a6ab-2d03d0e571db nodeName:}" failed. No retries permitted until 2026-09-18 18:45:17.590042041 +0000 UTC m=+53.809049599 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-q8mj9" (UID: "a7fa0c11-eafd-4961-a6ab-2d03d0e571db") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:13.791839 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:13.791798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:13.791839 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:13.791843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:13.792073 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.791960    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:45:13.792073 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.791985    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:45:13.792073 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.792030    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert podName:c9085e34-f699-4aec-9aa7-15b7a35766d7 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:17.792016724 +0000 UTC m=+54.011024279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-7r7xw" (UID: "c9085e34-f699-4aec-9aa7-15b7a35766d7") : secret "networking-console-plugin-cert" not found
Sep 18 18:45:13.792073 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:13.792043    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls podName:7a9675cf-d635-42ea-b0f6-dd3c65541273 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:17.79203664 +0000 UTC m=+54.011044195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-875v9" (UID: "7a9675cf-d635-42ea-b0f6-dd3c65541273") : secret "samples-operator-tls" not found
Sep 18 18:45:17.494323 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.494280    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vpnsw"]
Sep 18 18:45:17.496522 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.496503    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.499459 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.499425    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 18:45:17.500899 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.500870    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 18:45:17.501044 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.500899    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zff62\""
Sep 18 18:45:17.501044 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.500903    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 18:45:17.501044 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.500870    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 18:45:17.519951 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.519917    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vpnsw"]
Sep 18 18:45:17.621974 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.621938    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:17.622154 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.622022    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.622154 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.622051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7509e8dd-346c-461c-b2cd-caea0b5f793d-crio-socket\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.622154 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.622084    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7509e8dd-346c-461c-b2cd-caea0b5f793d-data-volume\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.622154 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:17.622101    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:17.622280 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:17.622199    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls podName:a7fa0c11-eafd-4961-a6ab-2d03d0e571db nodeName:}" failed. No retries permitted until 2026-09-18 18:45:25.622173309 +0000 UTC m=+61.841180865 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-q8mj9" (UID: "a7fa0c11-eafd-4961-a6ab-2d03d0e571db") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:17.622317 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.622279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.622362 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.622347    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rrpr\" (UniqueName: \"kubernetes.io/projected/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-api-access-6rrpr\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.722988 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.722944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.722988 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.722988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7509e8dd-346c-461c-b2cd-caea0b5f793d-crio-socket\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.723163 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.723011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7509e8dd-346c-461c-b2cd-caea0b5f793d-data-volume\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.723163 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:17.723091    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:45:17.723163 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.723131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.723290 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:17.723170    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:18.223153999 +0000 UTC m=+54.442161558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : configmap "kube-rbac-proxy" not found
Sep 18 18:45:17.723290 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:17.723225    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:17.723290 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.723227    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7509e8dd-346c-461c-b2cd-caea0b5f793d-crio-socket\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.723290 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:17.723279    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:18.223266844 +0000 UTC m=+54.442274400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:17.723290 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.723224    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rrpr\" (UniqueName: \"kubernetes.io/projected/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-api-access-6rrpr\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.723524 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.723344    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7509e8dd-346c-461c-b2cd-caea0b5f793d-data-volume\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.734869 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.734843    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rrpr\" (UniqueName: \"kubernetes.io/projected/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-api-access-6rrpr\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:17.824664 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.824570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:17.824664 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:17.824640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:17.824896 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:17.824759    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:45:17.824896 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:17.824762    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:45:17.824896 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:17.824823    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls podName:7a9675cf-d635-42ea-b0f6-dd3c65541273 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:25.824809114 +0000 UTC m=+62.043816669 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-875v9" (UID: "7a9675cf-d635-42ea-b0f6-dd3c65541273") : secret "samples-operator-tls" not found
Sep 18 18:45:17.824896 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:17.824837    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert podName:c9085e34-f699-4aec-9aa7-15b7a35766d7 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:25.824831057 +0000 UTC m=+62.043838612 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-7r7xw" (UID: "c9085e34-f699-4aec-9aa7-15b7a35766d7") : secret "networking-console-plugin-cert" not found
Sep 18 18:45:18.228201 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:18.228160    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:18.228386 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:18.228222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:18.228386 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:18.228287    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:45:18.228386 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:18.228313    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:18.228386 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:18.228348    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:19.228333819 +0000 UTC m=+55.447341375 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : configmap "kube-rbac-proxy" not found
Sep 18 18:45:18.228532 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:18.228417    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:19.228395459 +0000 UTC m=+55.447403018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:19.237030 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:19.236976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:19.237030 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:19.237044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:19.237459 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:19.237101    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:45:19.237459 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:19.237146    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:19.237459 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:19.237167    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:21.237153401 +0000 UTC m=+57.456160956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : configmap "kube-rbac-proxy" not found
Sep 18 18:45:19.237459 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:19.237185    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:21.237174209 +0000 UTC m=+57.456181766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:20.034716 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:20.034676    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bspq5_d2f53f4b-0b86-4f9a-a5f6-0c428539290f/dns-node-resolver/0.log"
Sep 18 18:45:21.232446 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:21.232419    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rqb8l_aa308550-2945-40b0-ba91-87a9c8541846/node-ca/0.log"
Sep 18 18:45:21.253510 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:21.253451    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:21.253894 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:21.253859    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:45:21.254008 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:21.253968    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:21.254111 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:21.254095    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:21.254189 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:21.254121    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:25.25407914 +0000 UTC m=+61.473086702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : configmap "kube-rbac-proxy" not found
Sep 18 18:45:21.254189 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:21.254166    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:25.254135487 +0000 UTC m=+61.473143060 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:22.236097 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:22.236068    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z4rg2_b51682e9-4379-4076-a853-ec0306df5d58/serve-healthcheck-canary/0.log"
Sep 18 18:45:22.833512 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:22.833479    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-x8vmt_cf71c2eb-68c0-4f63-94e8-d71050926938/kube-storage-version-migrator-operator/0.log"
Sep 18 18:45:24.494916 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:24.494883    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-86j9s"
Sep 18 18:45:25.287962 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:25.287911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:25.288159 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:25.287991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:25.288159 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:25.288037    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:45:25.288159 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:25.288120    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:25.288159 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:25.288122    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:33.288106082 +0000 UTC m=+69.507113637 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : configmap "kube-rbac-proxy" not found
Sep 18 18:45:25.288310 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:25.288183    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:33.288169845 +0000 UTC m=+69.507177409 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:25.691603 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:25.691559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:25.692002 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:25.691735    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:25.692002 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:25.691801    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls podName:a7fa0c11-eafd-4961-a6ab-2d03d0e571db nodeName:}" failed. No retries permitted until 2026-09-18 18:45:41.691786259 +0000 UTC m=+77.910793814 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-q8mj9" (UID: "a7fa0c11-eafd-4961-a6ab-2d03d0e571db") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:45:25.893139 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:25.893098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:25.893307 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:25.893191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:25.893307 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:25.893301    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:45:25.893374 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:25.893354    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert podName:c9085e34-f699-4aec-9aa7-15b7a35766d7 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:41.89334131 +0000 UTC m=+78.112348866 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-7r7xw" (UID: "c9085e34-f699-4aec-9aa7-15b7a35766d7") : secret "networking-console-plugin-cert" not found
Sep 18 18:45:25.895823 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:25.895790    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a9675cf-d635-42ea-b0f6-dd3c65541273-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-875v9\" (UID: \"7a9675cf-d635-42ea-b0f6-dd3c65541273\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:25.996926 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:25.996840    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-n5tz6\""
Sep 18 18:45:26.004616 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:26.004587    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"
Sep 18 18:45:26.285278 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:26.285197    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9"]
Sep 18 18:45:26.588145 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:26.588055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9" event={"ID":"7a9675cf-d635-42ea-b0f6-dd3c65541273","Type":"ContainerStarted","Data":"35c9cbd91aee7a593bf7f163cef37a655a32ce001648540a9680d573c4b90a85"}
Sep 18 18:45:28.598454 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:28.598415    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9" event={"ID":"7a9675cf-d635-42ea-b0f6-dd3c65541273","Type":"ContainerStarted","Data":"672311dbd427da10bd4442507b1b6c723afd50d21cf490c0da8035654d2c05a4"}
Sep 18 18:45:28.598941 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:28.598500    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9" event={"ID":"7a9675cf-d635-42ea-b0f6-dd3c65541273","Type":"ContainerStarted","Data":"a6628440953d7726cdf9623bc7f31590a78464fe93ffb2595db41e669ae85023"}
Sep 18 18:45:28.615877 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:28.615819    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-875v9" podStartSLOduration=16.948848045 podStartE2EDuration="18.615802238s" totalImagesPullingTime="1.666954193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:28.615337224 +0000 UTC m=+64.834344803" watchObservedRunningTime="2026-09-18 18:45:28.615802238 +0000 UTC m=+64.834809844"
Sep 18 18:45:29.525625 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:29.525578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:45:29.525842 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:29.525647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:45:29.528192 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:29.528166    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/946a2586-1164-4bdf-9348-22b7089112e0-metrics-tls\") pod \"dns-default-smkgk\" (UID: \"946a2586-1164-4bdf-9348-22b7089112e0\") " pod="openshift-dns/dns-default-smkgk"
Sep 18 18:45:29.528325 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:29.528299    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") pod \"image-registry-64c8d855c6-zrzwq\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") " pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:45:29.799505 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:29.799421    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m6jx4\""
Sep 18 18:45:29.807086 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:29.807052    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:45:29.815026 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:29.814999    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qf4nw\""
Sep 18 18:45:29.823373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:29.823328    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-smkgk"
Sep 18 18:45:30.029201 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.029159    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:45:30.029399 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.029226    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:45:30.032258 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.032233    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:45:30.032359 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.032271    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:45:30.039877 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:30.039841    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 18:45:30.039955 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:30.039943    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs podName:85ea9986-0a9c-4b8a-835b-f42a46dd8e55 nodeName:}" failed. No retries permitted until 2026-09-18 18:46:34.039921114 +0000 UTC m=+130.258928670 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs") pod "network-metrics-daemon-skwt5" (UID: "85ea9986-0a9c-4b8a-835b-f42a46dd8e55") : secret "metrics-daemon-secret" not found
Sep 18 18:45:30.042412 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.042378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dcfcf8b-4bd5-4919-9315-26fe6852ba7d-original-pull-secret\") pod \"global-pull-secret-syncer-zsrbq\" (UID: \"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d\") " pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:45:30.105805 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.105770    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64c8d855c6-zrzwq"]
Sep 18 18:45:30.108191 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:45:30.108156    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6273c63_ae77_482f_bee7_7399f9793173.slice/crio-a48bb4f545f1c22b620853f3589dfe3c0621a9b3a57b00523fbbe07cd6a6d6e3 WatchSource:0}: Error finding container a48bb4f545f1c22b620853f3589dfe3c0621a9b3a57b00523fbbe07cd6a6d6e3: Status 404 returned error can't find the container with id a48bb4f545f1c22b620853f3589dfe3c0621a9b3a57b00523fbbe07cd6a6d6e3
Sep 18 18:45:30.127114 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.127084    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-smkgk"]
Sep 18 18:45:30.130046 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:45:30.130018    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946a2586_1164_4bdf_9348_22b7089112e0.slice/crio-f698f1c599330dc463b725ffea83f2dcec102ca098174b691aa0bf0ac3bda642 WatchSource:0}: Error finding container f698f1c599330dc463b725ffea83f2dcec102ca098174b691aa0bf0ac3bda642: Status 404 returned error can't find the container with id f698f1c599330dc463b725ffea83f2dcec102ca098174b691aa0bf0ac3bda642
Sep 18 18:45:30.222834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.222808    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsrbq"
Sep 18 18:45:30.231082 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.231050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skwst\" (UniqueName: \"kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst\") pod \"network-check-target-9hxwx\" (UID: \"9a781944-d73f-4c37-8c50-8c71aaa9b2a6\") " pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:45:30.235022 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.234996    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 18:45:30.244547 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.244517    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 18:45:30.255054 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.255024    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skwst\" (UniqueName: \"kubernetes.io/projected/9a781944-d73f-4c37-8c50-8c71aaa9b2a6-kube-api-access-skwst\") pod \"network-check-target-9hxwx\" (UID: \"9a781944-d73f-4c37-8c50-8c71aaa9b2a6\") " pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:45:30.420400 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.420360    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vk59r\""
Sep 18 18:45:30.427890 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.427856    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:45:30.507181 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.507142    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zsrbq"]
Sep 18 18:45:30.511261 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:45:30.511226    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dcfcf8b_4bd5_4919_9315_26fe6852ba7d.slice/crio-51a050e41905721a2fcf47fd965cc50a4bd55dce0c5dce76462edef6e21a3295 WatchSource:0}: Error finding container 51a050e41905721a2fcf47fd965cc50a4bd55dce0c5dce76462edef6e21a3295: Status 404 returned error can't find the container with id 51a050e41905721a2fcf47fd965cc50a4bd55dce0c5dce76462edef6e21a3295
Sep 18 18:45:30.608066 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.608007    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-smkgk" event={"ID":"946a2586-1164-4bdf-9348-22b7089112e0","Type":"ContainerStarted","Data":"f698f1c599330dc463b725ffea83f2dcec102ca098174b691aa0bf0ac3bda642"}
Sep 18 18:45:30.609950 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.609919    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq" event={"ID":"e6273c63-ae77-482f-bee7-7399f9793173","Type":"ContainerStarted","Data":"773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016"}
Sep 18 18:45:30.610127 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.609981    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:45:30.610127 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.610002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq" event={"ID":"e6273c63-ae77-482f-bee7-7399f9793173","Type":"ContainerStarted","Data":"a48bb4f545f1c22b620853f3589dfe3c0621a9b3a57b00523fbbe07cd6a6d6e3"}
Sep 18 18:45:30.611361 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.611335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zsrbq" event={"ID":"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d","Type":"ContainerStarted","Data":"51a050e41905721a2fcf47fd965cc50a4bd55dce0c5dce76462edef6e21a3295"}
Sep 18 18:45:30.642575 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.642322    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq" podStartSLOduration=66.642297613 podStartE2EDuration="1m6.642297613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:30.641108645 +0000 UTC m=+66.860116224" watchObservedRunningTime="2026-09-18 18:45:30.642297613 +0000 UTC m=+66.861305192"
Sep 18 18:45:30.719128 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:30.719040    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9hxwx"]
Sep 18 18:45:30.723157 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:45:30.723126    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a781944_d73f_4c37_8c50_8c71aaa9b2a6.slice/crio-b5aab0a0926b08397990abb26a6e717f7b8cfdab74a8c3d610e416ef93f10b80 WatchSource:0}: Error finding container b5aab0a0926b08397990abb26a6e717f7b8cfdab74a8c3d610e416ef93f10b80: Status 404 returned error can't find the container with id b5aab0a0926b08397990abb26a6e717f7b8cfdab74a8c3d610e416ef93f10b80
Sep 18 18:45:31.618346 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:31.618302    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9hxwx" event={"ID":"9a781944-d73f-4c37-8c50-8c71aaa9b2a6","Type":"ContainerStarted","Data":"b5aab0a0926b08397990abb26a6e717f7b8cfdab74a8c3d610e416ef93f10b80"}
Sep 18 18:45:32.624218 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:32.624172    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-smkgk" event={"ID":"946a2586-1164-4bdf-9348-22b7089112e0","Type":"ContainerStarted","Data":"103a5138ef9ccedb526411098822f2e0c945044309c94da9e67ff85462f41100"}
Sep 18 18:45:32.624645 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:32.624241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-smkgk" event={"ID":"946a2586-1164-4bdf-9348-22b7089112e0","Type":"ContainerStarted","Data":"d64cd18e1dd16c6e157cf3f7f39402ab2f8e5f0b9af23af45af6018ceb4c360b"}
Sep 18 18:45:32.624645 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:32.624328    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-smkgk"
Sep 18 18:45:32.644696 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:32.644631    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-smkgk" podStartSLOduration=34.083330715 podStartE2EDuration="35.644607067s" totalImagesPullingTime="1.561276352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:32.644179687 +0000 UTC m=+68.863187261" watchObservedRunningTime="2026-09-18 18:45:32.644607067 +0000 UTC m=+68.863614646"
Sep 18 18:45:33.362693 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:33.362648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:33.362930 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:33.362732    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:33.363012 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:33.362928    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:33.363063 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:45:33.363017    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls podName:7509e8dd-346c-461c-b2cd-caea0b5f793d nodeName:}" failed. No retries permitted until 2026-09-18 18:45:49.362993776 +0000 UTC m=+85.582001347 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vpnsw" (UID: "7509e8dd-346c-461c-b2cd-caea0b5f793d") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:33.363600 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:33.363576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7509e8dd-346c-461c-b2cd-caea0b5f793d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:34.633045 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:34.633004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9hxwx" event={"ID":"9a781944-d73f-4c37-8c50-8c71aaa9b2a6","Type":"ContainerStarted","Data":"adcfd954364a106ac4b78093334236c83bf8438408a2d4361aaecf25f6fbf30d"}
Sep 18 18:45:34.633522 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:34.633161    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:45:34.655967 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:34.655919    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9hxwx" podStartSLOduration=66.851216499 podStartE2EDuration="1m10.655902255s" totalImagesPullingTime="3.804685756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:34.654080415 +0000 UTC m=+70.873087993" watchObservedRunningTime="2026-09-18 18:45:34.655902255 +0000 UTC m=+70.874909827"
Sep 18 18:45:37.645451 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:37.645356    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zsrbq" event={"ID":"1dcfcf8b-4bd5-4919-9315-26fe6852ba7d","Type":"ContainerStarted","Data":"4799ea36e1105fb24c834e9e3a429876e50ab784b85eee461699242b07a56d1d"}
Sep 18 18:45:37.662429 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:37.662365    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zsrbq" podStartSLOduration=67.001137271 podStartE2EDuration="1m13.662346093s" totalImagesPullingTime="6.661208822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:37.661390893 +0000 UTC m=+73.880398470" watchObservedRunningTime="2026-09-18 18:45:37.662346093 +0000 UTC m=+73.881353672"
Sep 18 18:45:41.731557 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:41.731510    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:41.734369 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:41.734334    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a7fa0c11-eafd-4961-a6ab-2d03d0e571db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-q8mj9\" (UID: \"a7fa0c11-eafd-4961-a6ab-2d03d0e571db\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:41.933105 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:41.933063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:41.935816 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:41.935779    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9085e34-f699-4aec-9aa7-15b7a35766d7-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-7r7xw\" (UID: \"c9085e34-f699-4aec-9aa7-15b7a35766d7\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:41.965794 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:41.965762    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nb7fx\""
Sep 18 18:45:41.973117 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:41.973088    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"
Sep 18 18:45:42.172382 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:42.172346    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jw2lz\""
Sep 18 18:45:42.180434 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:42.180407    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"
Sep 18 18:45:42.250218 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:45:42.250181    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7fa0c11_eafd_4961_a6ab_2d03d0e571db.slice/crio-85d665108ff996546a7b724043d02faf54889e2cd18612ef959bd9542e75b5da WatchSource:0}: Error finding container 85d665108ff996546a7b724043d02faf54889e2cd18612ef959bd9542e75b5da: Status 404 returned error can't find the container with id 85d665108ff996546a7b724043d02faf54889e2cd18612ef959bd9542e75b5da
Sep 18 18:45:42.250803 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:42.250750    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9"]
Sep 18 18:45:42.457987 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:42.457949    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw"]
Sep 18 18:45:42.460913 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:45:42.460883    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9085e34_f699_4aec_9aa7_15b7a35766d7.slice/crio-9765e86cee88ae2528ff63eef34c8eeac21100a288532c0d6dc9e1f9ca4fd7a3 WatchSource:0}: Error finding container 9765e86cee88ae2528ff63eef34c8eeac21100a288532c0d6dc9e1f9ca4fd7a3: Status 404 returned error can't find the container with id 9765e86cee88ae2528ff63eef34c8eeac21100a288532c0d6dc9e1f9ca4fd7a3
Sep 18 18:45:42.631342 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:42.631255    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-smkgk"
Sep 18 18:45:42.661622 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:42.661580    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw" event={"ID":"c9085e34-f699-4aec-9aa7-15b7a35766d7","Type":"ContainerStarted","Data":"9765e86cee88ae2528ff63eef34c8eeac21100a288532c0d6dc9e1f9ca4fd7a3"}
Sep 18 18:45:42.662939 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:42.662910    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9" event={"ID":"a7fa0c11-eafd-4961-a6ab-2d03d0e571db","Type":"ContainerStarted","Data":"85d665108ff996546a7b724043d02faf54889e2cd18612ef959bd9542e75b5da"}
Sep 18 18:45:43.917136 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:43.917097    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"]
Sep 18 18:45:43.921599 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:43.921567    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:43.925133 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:43.925100    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 18:45:43.926153 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:43.925634    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 18 18:45:43.926153 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:43.925925    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 18 18:45:43.926690 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:43.926665    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 18 18:45:43.926968 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:43.926939    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 18:45:43.927406 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:43.927228    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 18:45:43.927406 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:43.927324    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 18 18:45:43.940066 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:43.940031    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"]
Sep 18 18:45:44.047956 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.047915    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.047956 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.047956    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-ca\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.048202 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.047974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-hub\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.048202 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.048088    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.048202 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.048131    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc428696-d90d-44fa-a5e8-291e666fdbfc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.048327 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.048222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2kkz\" (UniqueName: \"kubernetes.io/projected/fc428696-d90d-44fa-a5e8-291e666fdbfc-kube-api-access-h2kkz\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.148954 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.148916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.148954 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.148954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-ca\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.149187 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.148977    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-hub\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.149187 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.149011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.149187 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.149039    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc428696-d90d-44fa-a5e8-291e666fdbfc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.149187 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.149102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2kkz\" (UniqueName: \"kubernetes.io/projected/fc428696-d90d-44fa-a5e8-291e666fdbfc-kube-api-access-h2kkz\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.150193 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.150146    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc428696-d90d-44fa-a5e8-291e666fdbfc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.152541 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.152511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-ca\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.152679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.152510    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.152679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.152586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.153446 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.153424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fc428696-d90d-44fa-a5e8-291e666fdbfc-hub\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.160288 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.160259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2kkz\" (UniqueName: \"kubernetes.io/projected/fc428696-d90d-44fa-a5e8-291e666fdbfc-kube-api-access-h2kkz\") pod \"cluster-proxy-proxy-agent-659bcc4547-mhzmg\" (UID: \"fc428696-d90d-44fa-a5e8-291e666fdbfc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.242181 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.242087    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"
Sep 18 18:45:44.654131 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.654056    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg"]
Sep 18 18:45:44.657744 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:45:44.657678    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc428696_d90d_44fa_a5e8_291e666fdbfc.slice/crio-71364f9b695c54c9afd908c5d1345a2af65b7e7bdf6ba454769217a112a9c36a WatchSource:0}: Error finding container 71364f9b695c54c9afd908c5d1345a2af65b7e7bdf6ba454769217a112a9c36a: Status 404 returned error can't find the container with id 71364f9b695c54c9afd908c5d1345a2af65b7e7bdf6ba454769217a112a9c36a
Sep 18 18:45:44.690009 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.689973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw" event={"ID":"c9085e34-f699-4aec-9aa7-15b7a35766d7","Type":"ContainerStarted","Data":"5c8a0dad639384e7cb8e75217b815dd25982298c2b210877fb9c6a4218dc92f5"}
Sep 18 18:45:44.691721 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.691649    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9" event={"ID":"a7fa0c11-eafd-4961-a6ab-2d03d0e571db","Type":"ContainerStarted","Data":"f6f5a4051e115892ab5fc6d79f8420f59101e07c7aa41d10480745bf3b75a255"}
Sep 18 18:45:44.692962 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.692938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg" event={"ID":"fc428696-d90d-44fa-a5e8-291e666fdbfc","Type":"ContainerStarted","Data":"71364f9b695c54c9afd908c5d1345a2af65b7e7bdf6ba454769217a112a9c36a"}
Sep 18 18:45:44.727762 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.726898    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-7r7xw" podStartSLOduration=33.539843252 podStartE2EDuration="34.726872797s" totalImagesPullingTime="1.187029545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:44.721547157 +0000 UTC m=+80.940554732" watchObservedRunningTime="2026-09-18 18:45:44.726872797 +0000 UTC m=+80.945880376"
Sep 18 18:45:44.760285 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:44.760172    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-q8mj9" podStartSLOduration=33.574768316 podStartE2EDuration="35.76014926s" totalImagesPullingTime="2.185380944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:44.759853157 +0000 UTC m=+80.978860738" watchObservedRunningTime="2026-09-18 18:45:44.76014926 +0000 UTC m=+80.979156834"
Sep 18 18:45:48.713174 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:48.713136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg" event={"ID":"fc428696-d90d-44fa-a5e8-291e666fdbfc","Type":"ContainerStarted","Data":"43d76d777c23310348e645ef255ec8839c059b7f7bcd30f5ff47f95e0b89524e"}
Sep 18 18:45:48.713572 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:48.713192    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg" event={"ID":"fc428696-d90d-44fa-a5e8-291e666fdbfc","Type":"ContainerStarted","Data":"6b9a64353f54ef86ae0b4dd57dba00118a3a573a1afd8eed05cd4b56652c36bc"}
Sep 18 18:45:49.398025 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:49.397973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:49.400555 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:49.400529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7509e8dd-346c-461c-b2cd-caea0b5f793d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vpnsw\" (UID: \"7509e8dd-346c-461c-b2cd-caea0b5f793d\") " pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:49.609867 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:49.609836    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zff62\""
Sep 18 18:45:49.616585 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:49.616556    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vpnsw"
Sep 18 18:45:49.719549 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:49.719513    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg" event={"ID":"fc428696-d90d-44fa-a5e8-291e666fdbfc","Type":"ContainerStarted","Data":"57f207682c57654345c7531ae1747d7368e3654ea97748cfb5385f3b52fb32d8"}
Sep 18 18:45:49.745722 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:49.745650    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-659bcc4547-mhzmg" podStartSLOduration=2.8874582699999998 podStartE2EDuration="6.745630611s" totalImagesPullingTime="3.858172341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:49.744379082 +0000 UTC m=+85.963386675" watchObservedRunningTime="2026-09-18 18:45:49.745630611 +0000 UTC m=+85.964638188"
Sep 18 18:45:49.812164 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:49.812130    2581 patch_prober.go:28] interesting pod/image-registry-64c8d855c6-zrzwq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 18:45:49.812325 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:49.812188    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq" podUID="e6273c63-ae77-482f-bee7-7399f9793173" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:45:49.892197 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:49.892160    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vpnsw"]
Sep 18 18:45:49.895760 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:45:49.895733    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7509e8dd_346c_461c_b2cd_caea0b5f793d.slice/crio-435b436a4b256fd1c1f40278421313f7ecdf3884849c88a310e7bbafbd495904 WatchSource:0}: Error finding container 435b436a4b256fd1c1f40278421313f7ecdf3884849c88a310e7bbafbd495904: Status 404 returned error can't find the container with id 435b436a4b256fd1c1f40278421313f7ecdf3884849c88a310e7bbafbd495904
Sep 18 18:45:50.725294 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:50.725246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vpnsw" event={"ID":"7509e8dd-346c-461c-b2cd-caea0b5f793d","Type":"ContainerStarted","Data":"5839749b171d845d89827d2ad3b0617b8574f756f87a229dd9c884e998409cd2"}
Sep 18 18:45:50.725674 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:50.725317    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vpnsw" event={"ID":"7509e8dd-346c-461c-b2cd-caea0b5f793d","Type":"ContainerStarted","Data":"13cabd1ffeca86519838ac833e3e32a754095419b50e438834e49e7fa3df62ac"}
Sep 18 18:45:50.725674 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:50.725346    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vpnsw" event={"ID":"7509e8dd-346c-461c-b2cd-caea0b5f793d","Type":"ContainerStarted","Data":"435b436a4b256fd1c1f40278421313f7ecdf3884849c88a310e7bbafbd495904"}
Sep 18 18:45:51.624492 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:51.624463    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:45:52.735069 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:52.735034    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vpnsw" event={"ID":"7509e8dd-346c-461c-b2cd-caea0b5f793d","Type":"ContainerStarted","Data":"584c0fc1835ba94882c40022efbdf56de8a8ba0746cf63da1280ab25919ec93b"}
Sep 18 18:45:52.759610 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:45:52.759553    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vpnsw" podStartSLOduration=33.867833413 podStartE2EDuration="35.759533338s" totalImagesPullingTime="1.891699925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:45:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:52.757696276 +0000 UTC m=+88.976703853" watchObservedRunningTime="2026-09-18 18:45:52.759533338 +0000 UTC m=+88.978540962"
Sep 18 18:46:05.639405 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.639372    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9hxwx"
Sep 18 18:46:05.675631 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.675593    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rwrc6"]
Sep 18 18:46:05.680918 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.680888    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.685785 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.685746    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 18:46:05.685998 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.685813    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 18:46:05.685998 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.685850    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 18:46:05.686160 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.686051    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-js7p5\""
Sep 18 18:46:05.686160 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.686128    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 18:46:05.829619 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.829589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t8m2\" (UniqueName: \"kubernetes.io/projected/761dd490-1d88-43f1-b0be-6aa71c53415b-kube-api-access-8t8m2\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.829619 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.829625    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/761dd490-1d88-43f1-b0be-6aa71c53415b-root\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.829871 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.829645    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-textfile\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.829871 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.829737    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/761dd490-1d88-43f1-b0be-6aa71c53415b-sys\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.829871 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.829778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/761dd490-1d88-43f1-b0be-6aa71c53415b-metrics-client-ca\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.829871 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.829807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.829871 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.829844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-tls\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.830040 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.829907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-wtmp\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.830040 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.829941    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-accelerators-collector-config\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.930688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.930648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t8m2\" (UniqueName: \"kubernetes.io/projected/761dd490-1d88-43f1-b0be-6aa71c53415b-kube-api-access-8t8m2\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.930900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.930723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/761dd490-1d88-43f1-b0be-6aa71c53415b-root\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.930900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.930757    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-textfile\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.930900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.930785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/761dd490-1d88-43f1-b0be-6aa71c53415b-sys\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.930900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.930798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/761dd490-1d88-43f1-b0be-6aa71c53415b-root\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.930900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.930814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/761dd490-1d88-43f1-b0be-6aa71c53415b-metrics-client-ca\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.930900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.930840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.930900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.930861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/761dd490-1d88-43f1-b0be-6aa71c53415b-sys\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.930900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.930882    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-tls\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.931341 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.930971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-wtmp\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.931341 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.931001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-accelerators-collector-config\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.931341 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.931150    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-textfile\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.931494 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.931374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/761dd490-1d88-43f1-b0be-6aa71c53415b-metrics-client-ca\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.931547 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.931521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-wtmp\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.931547 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.931522    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-accelerators-collector-config\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.933754 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.933692    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.933888 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.933855    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/761dd490-1d88-43f1-b0be-6aa71c53415b-node-exporter-tls\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.940064 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.940034    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t8m2\" (UniqueName: \"kubernetes.io/projected/761dd490-1d88-43f1-b0be-6aa71c53415b-kube-api-access-8t8m2\") pod \"node-exporter-rwrc6\" (UID: \"761dd490-1d88-43f1-b0be-6aa71c53415b\") " pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:05.993003 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:05.992963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rwrc6"
Sep 18 18:46:06.005310 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:46:06.005276    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod761dd490_1d88_43f1_b0be_6aa71c53415b.slice/crio-a21212c208992602e57f48064c10b44af01c7adecdf6049f3678e71a0f27bd73 WatchSource:0}: Error finding container a21212c208992602e57f48064c10b44af01c7adecdf6049f3678e71a0f27bd73: Status 404 returned error can't find the container with id a21212c208992602e57f48064c10b44af01c7adecdf6049f3678e71a0f27bd73
Sep 18 18:46:06.782653 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:06.782610    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwrc6" event={"ID":"761dd490-1d88-43f1-b0be-6aa71c53415b","Type":"ContainerStarted","Data":"a21212c208992602e57f48064c10b44af01c7adecdf6049f3678e71a0f27bd73"}
Sep 18 18:46:07.527390 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:07.527355    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64c8d855c6-zrzwq"]
Sep 18 18:46:07.788282 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:07.788174    2581 generic.go:356] "Generic (PLEG): container finished" podID="761dd490-1d88-43f1-b0be-6aa71c53415b" containerID="60250c904859a82120d69aa5f907dfa058be16552e7026fe30202c8e14dfc36d" exitCode=0
Sep 18 18:46:07.788282 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:07.788259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwrc6" event={"ID":"761dd490-1d88-43f1-b0be-6aa71c53415b","Type":"ContainerDied","Data":"60250c904859a82120d69aa5f907dfa058be16552e7026fe30202c8e14dfc36d"}
Sep 18 18:46:08.794494 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:08.794460    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwrc6" event={"ID":"761dd490-1d88-43f1-b0be-6aa71c53415b","Type":"ContainerStarted","Data":"1927c2a04dde2fdffa99f781faff324f5c99e822fcd6fd2bae1454ed4e4b2ae5"}
Sep 18 18:46:08.794911 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:08.794514    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwrc6" event={"ID":"761dd490-1d88-43f1-b0be-6aa71c53415b","Type":"ContainerStarted","Data":"0793b179eee6fec17783e5dfd422dd7295875c19fbf03a6ff664d1d9bcf91c63"}
Sep 18 18:46:08.817925 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:08.817851    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rwrc6" podStartSLOduration=2.666966438 podStartE2EDuration="3.817831493s" totalImagesPullingTime="1.074781255s" totalInitContainerRuntime="76.0838ms" isStatefulPod=true podCreationTimestamp="2026-09-18 18:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:08.817258098 +0000 UTC m=+105.036265675" watchObservedRunningTime="2026-09-18 18:46:08.817831493 +0000 UTC m=+105.036839147"
Sep 18 18:46:32.548007 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.547931    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq" podUID="e6273c63-ae77-482f-bee7-7399f9793173" containerName="registry" containerID="cri-o://773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016" gracePeriod=30
Sep 18 18:46:32.790302 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.790262    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:46:32.850743 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.850632    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-registry-certificates\") pod \"e6273c63-ae77-482f-bee7-7399f9793173\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") "
Sep 18 18:46:32.850743 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.850688    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk996\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-kube-api-access-tk996\") pod \"e6273c63-ae77-482f-bee7-7399f9793173\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") "
Sep 18 18:46:32.850743 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.850743    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6273c63-ae77-482f-bee7-7399f9793173-ca-trust-extracted\") pod \"e6273c63-ae77-482f-bee7-7399f9793173\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") "
Sep 18 18:46:32.851016 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.850762    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-trusted-ca\") pod \"e6273c63-ae77-482f-bee7-7399f9793173\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") "
Sep 18 18:46:32.851016 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.850786    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") pod \"e6273c63-ae77-482f-bee7-7399f9793173\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") "
Sep 18 18:46:32.851016 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.850829    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-installation-pull-secrets\") pod \"e6273c63-ae77-482f-bee7-7399f9793173\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") "
Sep 18 18:46:32.851016 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.850851    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-bound-sa-token\") pod \"e6273c63-ae77-482f-bee7-7399f9793173\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") "
Sep 18 18:46:32.851016 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.850878    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-image-registry-private-configuration\") pod \"e6273c63-ae77-482f-bee7-7399f9793173\" (UID: \"e6273c63-ae77-482f-bee7-7399f9793173\") "
Sep 18 18:46:32.851262 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.851152    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e6273c63-ae77-482f-bee7-7399f9793173" (UID: "e6273c63-ae77-482f-bee7-7399f9793173"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:46:32.851643 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.851590    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e6273c63-ae77-482f-bee7-7399f9793173" (UID: "e6273c63-ae77-482f-bee7-7399f9793173"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:46:32.853599 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.853532    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e6273c63-ae77-482f-bee7-7399f9793173" (UID: "e6273c63-ae77-482f-bee7-7399f9793173"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:46:32.853740 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.853682    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e6273c63-ae77-482f-bee7-7399f9793173" (UID: "e6273c63-ae77-482f-bee7-7399f9793173"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:46:32.853862 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.853801    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-kube-api-access-tk996" (OuterVolumeSpecName: "kube-api-access-tk996") pod "e6273c63-ae77-482f-bee7-7399f9793173" (UID: "e6273c63-ae77-482f-bee7-7399f9793173"). InnerVolumeSpecName "kube-api-access-tk996". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:46:32.853909 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.853869    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e6273c63-ae77-482f-bee7-7399f9793173" (UID: "e6273c63-ae77-482f-bee7-7399f9793173"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:46:32.854007 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.853980    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e6273c63-ae77-482f-bee7-7399f9793173" (UID: "e6273c63-ae77-482f-bee7-7399f9793173"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:46:32.860115 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.860078    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6273c63-ae77-482f-bee7-7399f9793173-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e6273c63-ae77-482f-bee7-7399f9793173" (UID: "e6273c63-ae77-482f-bee7-7399f9793173"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:46:32.878905 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.878865    2581 generic.go:356] "Generic (PLEG): container finished" podID="e6273c63-ae77-482f-bee7-7399f9793173" containerID="773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016" exitCode=0
Sep 18 18:46:32.879095 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.878945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq" event={"ID":"e6273c63-ae77-482f-bee7-7399f9793173","Type":"ContainerDied","Data":"773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016"}
Sep 18 18:46:32.879095 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.878982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq" event={"ID":"e6273c63-ae77-482f-bee7-7399f9793173","Type":"ContainerDied","Data":"a48bb4f545f1c22b620853f3589dfe3c0621a9b3a57b00523fbbe07cd6a6d6e3"}
Sep 18 18:46:32.879095 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.878980    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64c8d855c6-zrzwq"
Sep 18 18:46:32.879095 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.879068    2581 scope.go:117] "RemoveContainer" containerID="773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016"
Sep 18 18:46:32.887984 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.887961    2581 scope.go:117] "RemoveContainer" containerID="773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016"
Sep 18 18:46:32.888368 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:46:32.888342    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016\": container with ID starting with 773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016 not found: ID does not exist" containerID="773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016"
Sep 18 18:46:32.888430 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.888384    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016"} err="failed to get container status \"773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016\": rpc error: code = NotFound desc = could not find container \"773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016\": container with ID starting with 773288b5c20d701ac5cca3b9cb155feb7e6d229a01a5da3e750c3966fa13b016 not found: ID does not exist"
Sep 18 18:46:32.902159 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.902119    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64c8d855c6-zrzwq"]
Sep 18 18:46:32.907913 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.907881    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64c8d855c6-zrzwq"]
Sep 18 18:46:32.951606 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.951568    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-registry-certificates\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:46:32.951606 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.951602    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk996\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-kube-api-access-tk996\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:46:32.951842 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.951616    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6273c63-ae77-482f-bee7-7399f9793173-ca-trust-extracted\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:46:32.951842 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.951629    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6273c63-ae77-482f-bee7-7399f9793173-trusted-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:46:32.951842 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.951641    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-registry-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:46:32.951842 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.951651    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-installation-pull-secrets\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:46:32.951842 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.951662    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6273c63-ae77-482f-bee7-7399f9793173-bound-sa-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:46:32.951842 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:32.951674    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6273c63-ae77-482f-bee7-7399f9793173-image-registry-private-configuration\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:46:34.059695 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:34.059636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:46:34.062308 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:34.062277    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ea9986-0a9c-4b8a-835b-f42a46dd8e55-metrics-certs\") pod \"network-metrics-daemon-skwt5\" (UID: \"85ea9986-0a9c-4b8a-835b-f42a46dd8e55\") " pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:46:34.119691 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:34.119649    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xdsv4\""
Sep 18 18:46:34.128157 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:34.128121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skwt5"
Sep 18 18:46:34.311642 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:34.311551    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6273c63-ae77-482f-bee7-7399f9793173" path="/var/lib/kubelet/pods/e6273c63-ae77-482f-bee7-7399f9793173/volumes"
Sep 18 18:46:34.411414 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:34.411378    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-skwt5"]
Sep 18 18:46:34.415575 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:46:34.414430    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ea9986_0a9c_4b8a_835b_f42a46dd8e55.slice/crio-423e4787ce60af1c5635e6dfea1f8ed69aaf223e69585663b07052873c5a2ea3 WatchSource:0}: Error finding container 423e4787ce60af1c5635e6dfea1f8ed69aaf223e69585663b07052873c5a2ea3: Status 404 returned error can't find the container with id 423e4787ce60af1c5635e6dfea1f8ed69aaf223e69585663b07052873c5a2ea3
Sep 18 18:46:34.889559 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:34.889511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-skwt5" event={"ID":"85ea9986-0a9c-4b8a-835b-f42a46dd8e55","Type":"ContainerStarted","Data":"423e4787ce60af1c5635e6dfea1f8ed69aaf223e69585663b07052873c5a2ea3"}
Sep 18 18:46:35.895117 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:35.895083    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-skwt5" event={"ID":"85ea9986-0a9c-4b8a-835b-f42a46dd8e55","Type":"ContainerStarted","Data":"e841ea8663bd91c1d80e912e48082383b86bdc4f3904eb7244575594decefade"}
Sep 18 18:46:35.895504 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:35.895133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-skwt5" event={"ID":"85ea9986-0a9c-4b8a-835b-f42a46dd8e55","Type":"ContainerStarted","Data":"a165a2255a68d3bd326e3c99173ba7dcc6844bd9ad73666b1a2aec323444051f"}
Sep 18 18:46:39.913161 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:39.913123    2581 generic.go:356] "Generic (PLEG): container finished" podID="cf71c2eb-68c0-4f63-94e8-d71050926938" containerID="8ae5be1cb6607ce18b9429950a3339612eed052953f00e839b27345f4dee71c1" exitCode=0
Sep 18 18:46:39.913577 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:39.913201    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt" event={"ID":"cf71c2eb-68c0-4f63-94e8-d71050926938","Type":"ContainerDied","Data":"8ae5be1cb6607ce18b9429950a3339612eed052953f00e839b27345f4dee71c1"}
Sep 18 18:46:39.913622 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:39.913591    2581 scope.go:117] "RemoveContainer" containerID="8ae5be1cb6607ce18b9429950a3339612eed052953f00e839b27345f4dee71c1"
Sep 18 18:46:39.932506 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:39.932448    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-skwt5" podStartSLOduration=135.015120582 podStartE2EDuration="2m15.932430091s" totalImagesPullingTime="917.309509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:35.913759685 +0000 UTC m=+132.132767262" watchObservedRunningTime="2026-09-18 18:46:39.932430091 +0000 UTC m=+136.151437676"
Sep 18 18:46:40.918821 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:46:40.918772    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-x8vmt" event={"ID":"cf71c2eb-68c0-4f63-94e8-d71050926938","Type":"ContainerStarted","Data":"1a974e6d96224f0512b61a2ee258b596c440ffa82ba56894fdfa3f9034092b1a"}
Sep 18 18:49:24.210591 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:49:24.210556    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 18:50:08.010263 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.010216    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"]
Sep 18 18:50:08.010751 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.010542    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6273c63-ae77-482f-bee7-7399f9793173" containerName="registry"
Sep 18 18:50:08.010751 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.010559    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6273c63-ae77-482f-bee7-7399f9793173" containerName="registry"
Sep 18 18:50:08.010751 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.010615    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6273c63-ae77-482f-bee7-7399f9793173" containerName="registry"
Sep 18 18:50:08.013487 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.013469    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"
Sep 18 18:50:08.019127 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.019096    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 18 18:50:08.019127 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.019119    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-vs6pj\""
Sep 18 18:50:08.019343 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.019199    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:50:08.034666 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.034633    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"]
Sep 18 18:50:08.071506 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.071463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njkqp\" (UniqueName: \"kubernetes.io/projected/a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a-kube-api-access-njkqp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hnbb\" (UID: \"a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"
Sep 18 18:50:08.071688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.071525    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hnbb\" (UID: \"a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"
Sep 18 18:50:08.172599 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.172561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njkqp\" (UniqueName: \"kubernetes.io/projected/a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a-kube-api-access-njkqp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hnbb\" (UID: \"a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"
Sep 18 18:50:08.172783 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.172630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hnbb\" (UID: \"a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"
Sep 18 18:50:08.172978 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.172962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hnbb\" (UID: \"a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"
Sep 18 18:50:08.182815 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.182788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njkqp\" (UniqueName: \"kubernetes.io/projected/a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a-kube-api-access-njkqp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2hnbb\" (UID: \"a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"
Sep 18 18:50:08.322282 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.322185    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"
Sep 18 18:50:08.603674 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.603648    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb"]
Sep 18 18:50:08.605732 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:50:08.605682    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b67fb3_75d7_48f9_a2c6_0fd463b48a8a.slice/crio-10f53b183c726589973cbeef7802c2050c838834d6bfffc6fc5fac7949ba5d5f WatchSource:0}: Error finding container 10f53b183c726589973cbeef7802c2050c838834d6bfffc6fc5fac7949ba5d5f: Status 404 returned error can't find the container with id 10f53b183c726589973cbeef7802c2050c838834d6bfffc6fc5fac7949ba5d5f
Sep 18 18:50:08.608286 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.608266    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:50:08.625221 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:08.625186    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb" event={"ID":"a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a","Type":"ContainerStarted","Data":"10f53b183c726589973cbeef7802c2050c838834d6bfffc6fc5fac7949ba5d5f"}
Sep 18 18:50:11.638881 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:11.638839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb" event={"ID":"a7b67fb3-75d7-48f9-a2c6-0fd463b48a8a","Type":"ContainerStarted","Data":"563332f1b95faae94e5faf733f3f84dd8c2e711afd888ea82a9b549298602996"}
Sep 18 18:50:12.359192 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.359129    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2hnbb" podStartSLOduration=2.819234041 podStartE2EDuration="5.359109418s" totalImagesPullingTime="2.539875377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:50:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:50:11.682455398 +0000 UTC m=+347.901462976" watchObservedRunningTime="2026-09-18 18:50:12.359109418 +0000 UTC m=+348.578117004"
Sep 18 18:50:12.359668 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.359647    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"]
Sep 18 18:50:12.363089 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.363062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:12.366041 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.366014    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 18:50:12.366185 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.366099    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 18:50:12.367339 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.367320    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mxnzd\""
Sep 18 18:50:12.378345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.378311    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"]
Sep 18 18:50:12.403623 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.403587    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:12.403847 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.403640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbgq5\" (UniqueName: \"kubernetes.io/projected/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-kube-api-access-qbgq5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:12.403847 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.403664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:12.504299 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.504262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:12.504498 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.504316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbgq5\" (UniqueName: \"kubernetes.io/projected/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-kube-api-access-qbgq5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:12.504498 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.504342    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:12.504696 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.504675    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:12.504785 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.504722    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:12.530370 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.530339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbgq5\" (UniqueName: \"kubernetes.io/projected/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-kube-api-access-qbgq5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:12.672320 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:12.672289    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:13.003621 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:13.003547    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"]
Sep 18 18:50:13.006838 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:50:13.006797    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf2fbaf_076f_4cfe_9b02_4ba631ab7864.slice/crio-9f68ae564a2e61180625bc47a90ee608e5cc63ffc181f94f1245907b245b2532 WatchSource:0}: Error finding container 9f68ae564a2e61180625bc47a90ee608e5cc63ffc181f94f1245907b245b2532: Status 404 returned error can't find the container with id 9f68ae564a2e61180625bc47a90ee608e5cc63ffc181f94f1245907b245b2532
Sep 18 18:50:13.647693 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:13.647648    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4" event={"ID":"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864","Type":"ContainerStarted","Data":"9f68ae564a2e61180625bc47a90ee608e5cc63ffc181f94f1245907b245b2532"}
Sep 18 18:50:19.672481 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:19.672440    2581 generic.go:356] "Generic (PLEG): container finished" podID="8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" containerID="fee6b7a6c484fb81c42661061c375428c1d90c8701e5199742f0193333e8d9df" exitCode=0
Sep 18 18:50:19.672943 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:19.672527    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4" event={"ID":"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864","Type":"ContainerDied","Data":"fee6b7a6c484fb81c42661061c375428c1d90c8701e5199742f0193333e8d9df"}
Sep 18 18:50:21.878130 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:21.878092    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dkkpr"]
Sep 18 18:50:21.887219 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:21.887182    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dkkpr"
Sep 18 18:50:21.890669 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:21.890638    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7l6sj\""
Sep 18 18:50:21.891018 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:21.890645    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 18:50:21.891690 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:21.891654    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 18:50:21.893631 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:21.893595    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dkkpr"]
Sep 18 18:50:21.981954 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:21.981911    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvb4b\" (UniqueName: \"kubernetes.io/projected/e19a6e30-8674-402c-9e6b-e848fed54ecc-kube-api-access-rvb4b\") pod \"cert-manager-55b4d954d5-dkkpr\" (UID: \"e19a6e30-8674-402c-9e6b-e848fed54ecc\") " pod="cert-manager/cert-manager-55b4d954d5-dkkpr"
Sep 18 18:50:21.982151 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:21.981981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e19a6e30-8674-402c-9e6b-e848fed54ecc-bound-sa-token\") pod \"cert-manager-55b4d954d5-dkkpr\" (UID: \"e19a6e30-8674-402c-9e6b-e848fed54ecc\") " pod="cert-manager/cert-manager-55b4d954d5-dkkpr"
Sep 18 18:50:22.083220 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:22.083175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvb4b\" (UniqueName: \"kubernetes.io/projected/e19a6e30-8674-402c-9e6b-e848fed54ecc-kube-api-access-rvb4b\") pod \"cert-manager-55b4d954d5-dkkpr\" (UID: \"e19a6e30-8674-402c-9e6b-e848fed54ecc\") " pod="cert-manager/cert-manager-55b4d954d5-dkkpr"
Sep 18 18:50:22.083402 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:22.083261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e19a6e30-8674-402c-9e6b-e848fed54ecc-bound-sa-token\") pod \"cert-manager-55b4d954d5-dkkpr\" (UID: \"e19a6e30-8674-402c-9e6b-e848fed54ecc\") " pod="cert-manager/cert-manager-55b4d954d5-dkkpr"
Sep 18 18:50:22.098148 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:22.098113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e19a6e30-8674-402c-9e6b-e848fed54ecc-bound-sa-token\") pod \"cert-manager-55b4d954d5-dkkpr\" (UID: \"e19a6e30-8674-402c-9e6b-e848fed54ecc\") " pod="cert-manager/cert-manager-55b4d954d5-dkkpr"
Sep 18 18:50:22.098510 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:22.098486    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvb4b\" (UniqueName: \"kubernetes.io/projected/e19a6e30-8674-402c-9e6b-e848fed54ecc-kube-api-access-rvb4b\") pod \"cert-manager-55b4d954d5-dkkpr\" (UID: \"e19a6e30-8674-402c-9e6b-e848fed54ecc\") " pod="cert-manager/cert-manager-55b4d954d5-dkkpr"
Sep 18 18:50:22.199370 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:22.199335    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dkkpr"
Sep 18 18:50:22.470272 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:22.470247    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dkkpr"]
Sep 18 18:50:22.472330 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:50:22.472305    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode19a6e30_8674_402c_9e6b_e848fed54ecc.slice/crio-1f036693e763022fb3b030ef7f057765dcdf3eeb625db8615f6f01636a0bb07a WatchSource:0}: Error finding container 1f036693e763022fb3b030ef7f057765dcdf3eeb625db8615f6f01636a0bb07a: Status 404 returned error can't find the container with id 1f036693e763022fb3b030ef7f057765dcdf3eeb625db8615f6f01636a0bb07a
Sep 18 18:50:22.686784 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:22.686747    2581 generic.go:356] "Generic (PLEG): container finished" podID="8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" containerID="33af3e733f761fb48188b6637b69b2ad7424aea1a844b9ed51cf846cbab7dd6e" exitCode=0
Sep 18 18:50:22.686956 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:22.686859    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4" event={"ID":"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864","Type":"ContainerDied","Data":"33af3e733f761fb48188b6637b69b2ad7424aea1a844b9ed51cf846cbab7dd6e"}
Sep 18 18:50:22.688064 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:22.688039    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dkkpr" event={"ID":"e19a6e30-8674-402c-9e6b-e848fed54ecc","Type":"ContainerStarted","Data":"1f036693e763022fb3b030ef7f057765dcdf3eeb625db8615f6f01636a0bb07a"}
Sep 18 18:50:29.720306 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:29.720273    2581 generic.go:356] "Generic (PLEG): container finished" podID="8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" containerID="84d3df8b3bbddb5406801266db59cd46ef533c48fb5d4042ac681373d9b1623b" exitCode=0
Sep 18 18:50:29.720735 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:29.720327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4" event={"ID":"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864","Type":"ContainerDied","Data":"84d3df8b3bbddb5406801266db59cd46ef533c48fb5d4042ac681373d9b1623b"}
Sep 18 18:50:30.855962 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:30.855924    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:30.955374 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:30.955343    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-util\") pod \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") "
Sep 18 18:50:30.955547 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:30.955395    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbgq5\" (UniqueName: \"kubernetes.io/projected/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-kube-api-access-qbgq5\") pod \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") "
Sep 18 18:50:30.955547 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:30.955416    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-bundle\") pod \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\" (UID: \"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864\") "
Sep 18 18:50:30.955907 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:30.955874    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-bundle" (OuterVolumeSpecName: "bundle") pod "8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" (UID: "8cf2fbaf-076f-4cfe-9b02-4ba631ab7864"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:50:30.957825 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:30.957798    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-kube-api-access-qbgq5" (OuterVolumeSpecName: "kube-api-access-qbgq5") pod "8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" (UID: "8cf2fbaf-076f-4cfe-9b02-4ba631ab7864"). InnerVolumeSpecName "kube-api-access-qbgq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:50:30.961588 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:30.961559    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-util" (OuterVolumeSpecName: "util") pod "8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" (UID: "8cf2fbaf-076f-4cfe-9b02-4ba631ab7864"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:50:31.056387 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:31.056289    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbgq5\" (UniqueName: \"kubernetes.io/projected/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-kube-api-access-qbgq5\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:50:31.056387 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:31.056326    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-bundle\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:50:31.056387 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:31.056335    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cf2fbaf-076f-4cfe-9b02-4ba631ab7864-util\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:50:31.729672 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:31.729639    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4"
Sep 18 18:50:31.729880 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:31.729642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8mzm4" event={"ID":"8cf2fbaf-076f-4cfe-9b02-4ba631ab7864","Type":"ContainerDied","Data":"9f68ae564a2e61180625bc47a90ee608e5cc63ffc181f94f1245907b245b2532"}
Sep 18 18:50:31.729880 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:31.729761    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f68ae564a2e61180625bc47a90ee608e5cc63ffc181f94f1245907b245b2532"
Sep 18 18:50:34.968310 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.968271    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"]
Sep 18 18:50:34.968747 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.968530    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" containerName="util"
Sep 18 18:50:34.968747 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.968542    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" containerName="util"
Sep 18 18:50:34.968747 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.968549    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" containerName="pull"
Sep 18 18:50:34.968747 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.968555    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" containerName="pull"
Sep 18 18:50:34.968747 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.968569    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" containerName="extract"
Sep 18 18:50:34.968747 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.968575    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" containerName="extract"
Sep 18 18:50:34.968747 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.968620    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf2fbaf-076f-4cfe-9b02-4ba631ab7864" containerName="extract"
Sep 18 18:50:34.972161 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.972137    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"
Sep 18 18:50:34.977026 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.976997    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-drwg4\""
Sep 18 18:50:34.977194 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.976997    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 18:50:34.977194 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.977001    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:50:34.982067 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:34.982038    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"]
Sep 18 18:50:35.085827 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:35.085788    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc6ac864-bc6c-4ef2-b908-b58fc191a053-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2dlbs\" (UID: \"cc6ac864-bc6c-4ef2-b908-b58fc191a053\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"
Sep 18 18:50:35.085827 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:35.085836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67g9g\" (UniqueName: \"kubernetes.io/projected/cc6ac864-bc6c-4ef2-b908-b58fc191a053-kube-api-access-67g9g\") pod \"openshift-lws-operator-fd8ccff4c-2dlbs\" (UID: \"cc6ac864-bc6c-4ef2-b908-b58fc191a053\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"
Sep 18 18:50:35.186524 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:35.186483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc6ac864-bc6c-4ef2-b908-b58fc191a053-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2dlbs\" (UID: \"cc6ac864-bc6c-4ef2-b908-b58fc191a053\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"
Sep 18 18:50:35.186735 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:35.186531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67g9g\" (UniqueName: \"kubernetes.io/projected/cc6ac864-bc6c-4ef2-b908-b58fc191a053-kube-api-access-67g9g\") pod \"openshift-lws-operator-fd8ccff4c-2dlbs\" (UID: \"cc6ac864-bc6c-4ef2-b908-b58fc191a053\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"
Sep 18 18:50:35.186971 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:35.186949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc6ac864-bc6c-4ef2-b908-b58fc191a053-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2dlbs\" (UID: \"cc6ac864-bc6c-4ef2-b908-b58fc191a053\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"
Sep 18 18:50:35.196319 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:35.196292    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67g9g\" (UniqueName: \"kubernetes.io/projected/cc6ac864-bc6c-4ef2-b908-b58fc191a053-kube-api-access-67g9g\") pod \"openshift-lws-operator-fd8ccff4c-2dlbs\" (UID: \"cc6ac864-bc6c-4ef2-b908-b58fc191a053\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"
Sep 18 18:50:35.282279 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:35.282182    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"
Sep 18 18:50:35.556638 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:35.556609    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs"]
Sep 18 18:50:35.559330 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:50:35.559302    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc6ac864_bc6c_4ef2_b908_b58fc191a053.slice/crio-94e3f5e37aeade23ea9b9bff366819f009e0ef1d736e73321f7d11c40573c137 WatchSource:0}: Error finding container 94e3f5e37aeade23ea9b9bff366819f009e0ef1d736e73321f7d11c40573c137: Status 404 returned error can't find the container with id 94e3f5e37aeade23ea9b9bff366819f009e0ef1d736e73321f7d11c40573c137
Sep 18 18:50:35.746640 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:35.746596    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs" event={"ID":"cc6ac864-bc6c-4ef2-b908-b58fc191a053","Type":"ContainerStarted","Data":"94e3f5e37aeade23ea9b9bff366819f009e0ef1d736e73321f7d11c40573c137"}
Sep 18 18:50:41.771668 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:41.771623    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs" event={"ID":"cc6ac864-bc6c-4ef2-b908-b58fc191a053","Type":"ContainerStarted","Data":"4e458ca6b890f3ec41cf0e1f251845d238c7fc9ace085faf06ff9b027801d17f"}
Sep 18 18:50:41.790059 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:41.789995    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dlbs" podStartSLOduration=2.427136958 podStartE2EDuration="7.789978244s" totalImagesPullingTime="5.362841286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:50:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:50:41.789450632 +0000 UTC m=+378.008458211" watchObservedRunningTime="2026-09-18 18:50:41.789978244 +0000 UTC m=+378.008985813"
Sep 18 18:50:47.799962 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:47.799923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dkkpr" event={"ID":"e19a6e30-8674-402c-9e6b-e848fed54ecc","Type":"ContainerStarted","Data":"f32c076b2da8b986e7e5ab646759993e7404d6918bebd0bf36f10c485742ae14"}
Sep 18 18:50:47.820099 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:50:47.820043    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-dkkpr" podStartSLOduration=2.079557744 podStartE2EDuration="26.820024569s" totalImagesPullingTime="24.740466825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:50:47.819888738 +0000 UTC m=+384.038896329" watchObservedRunningTime="2026-09-18 18:50:47.820024569 +0000 UTC m=+384.039032146"
Sep 18 18:51:02.679698 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.679652    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"]
Sep 18 18:51:02.723091 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.723042    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"]
Sep 18 18:51:02.723256 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.723172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.728687 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.728656    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 18:51:02.729433 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.729411    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-t6jkl\""
Sep 18 18:51:02.740518 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.740486    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 18:51:02.750196 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.750167    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 18:51:02.750347 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.750218    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 18:51:02.797251 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.797219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a923edf-fadc-4e15-95d7-e09df8794c9e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.797251 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.797257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a923edf-fadc-4e15-95d7-e09df8794c9e-webhook-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.797524 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.797293    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a923edf-fadc-4e15-95d7-e09df8794c9e-tmp\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.797524 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.797393    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a923edf-fadc-4e15-95d7-e09df8794c9e-manifests\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.797524 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.797438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2crv\" (UniqueName: \"kubernetes.io/projected/4a923edf-fadc-4e15-95d7-e09df8794c9e-kube-api-access-w2crv\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.897909 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.897805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2crv\" (UniqueName: \"kubernetes.io/projected/4a923edf-fadc-4e15-95d7-e09df8794c9e-kube-api-access-w2crv\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.897909 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.897865    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a923edf-fadc-4e15-95d7-e09df8794c9e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.897909 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.897893    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a923edf-fadc-4e15-95d7-e09df8794c9e-webhook-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.898198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.897919    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a923edf-fadc-4e15-95d7-e09df8794c9e-tmp\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.898198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.897939    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a923edf-fadc-4e15-95d7-e09df8794c9e-manifests\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.898303 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.898256    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a923edf-fadc-4e15-95d7-e09df8794c9e-manifests\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.900390 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.900347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a923edf-fadc-4e15-95d7-e09df8794c9e-tmp\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.900643 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.900621    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a923edf-fadc-4e15-95d7-e09df8794c9e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.900785 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.900625    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a923edf-fadc-4e15-95d7-e09df8794c9e-webhook-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:02.908367 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:02.908330    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2crv\" (UniqueName: \"kubernetes.io/projected/4a923edf-fadc-4e15-95d7-e09df8794c9e-kube-api-access-w2crv\") pod \"opendatahub-operator-controller-manager-699fb7c78d-mcfsk\" (UID: \"4a923edf-fadc-4e15-95d7-e09df8794c9e\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:03.033903 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:03.033844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:03.338270 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:03.338127    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"]
Sep 18 18:51:03.340768 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:51:03.340733    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a923edf_fadc_4e15_95d7_e09df8794c9e.slice/crio-14f69740366eeeefb038803679597497e56bcbb32d82510dd8d1fdf8d1fba932 WatchSource:0}: Error finding container 14f69740366eeeefb038803679597497e56bcbb32d82510dd8d1fdf8d1fba932: Status 404 returned error can't find the container with id 14f69740366eeeefb038803679597497e56bcbb32d82510dd8d1fdf8d1fba932
Sep 18 18:51:03.867216 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:03.867174    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk" event={"ID":"4a923edf-fadc-4e15-95d7-e09df8794c9e","Type":"ContainerStarted","Data":"14f69740366eeeefb038803679597497e56bcbb32d82510dd8d1fdf8d1fba932"}
Sep 18 18:51:06.884297 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:06.884247    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk" event={"ID":"4a923edf-fadc-4e15-95d7-e09df8794c9e","Type":"ContainerStarted","Data":"97380898ef37b367f66e65b8d5133827eb350c2c8e28e9e9853880107698dc69"}
Sep 18 18:51:07.891039 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:07.890945    2581 generic.go:356] "Generic (PLEG): container finished" podID="4a923edf-fadc-4e15-95d7-e09df8794c9e" containerID="97380898ef37b367f66e65b8d5133827eb350c2c8e28e9e9853880107698dc69" exitCode=0
Sep 18 18:51:07.891039 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:07.890991    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk" event={"ID":"4a923edf-fadc-4e15-95d7-e09df8794c9e","Type":"ContainerDied","Data":"97380898ef37b367f66e65b8d5133827eb350c2c8e28e9e9853880107698dc69"}
Sep 18 18:51:08.897522 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:08.897479    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk" event={"ID":"4a923edf-fadc-4e15-95d7-e09df8794c9e","Type":"ContainerStarted","Data":"ce7788d1fa47c4ce06d58a28487ac0ada48652bab765db17cf7e577999827e87"}
Sep 18 18:51:08.898002 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:08.897639    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:08.931721 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:08.931646    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk" podStartSLOduration=2.561233408 podStartE2EDuration="6.931621703s" totalImagesPullingTime="3.345611273s" totalInitContainerRuntime="1.024777022s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:08.929321944 +0000 UTC m=+405.148329531" watchObservedRunningTime="2026-09-18 18:51:08.931621703 +0000 UTC m=+405.150629282"
Sep 18 18:51:19.903783 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:19.903739    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-mcfsk"
Sep 18 18:51:24.012148 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.012097    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"]
Sep 18 18:51:24.015008 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.014980    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.018448 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.018410    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 18 18:51:24.018448 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.018448    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 18:51:24.018688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.018478    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 18:51:24.018688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.018595    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 18:51:24.019600 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.019582    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-v2zvp\""
Sep 18 18:51:24.027227 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.027192    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"]
Sep 18 18:51:24.057356 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.057309    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-tls-certs\") pod \"kube-auth-proxy-6cd84dd4ff-z94r4\" (UID: \"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6\") " pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.057566 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.057368    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4pdn\" (UniqueName: \"kubernetes.io/projected/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-kube-api-access-p4pdn\") pod \"kube-auth-proxy-6cd84dd4ff-z94r4\" (UID: \"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6\") " pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.057566 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.057398    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-tmp\") pod \"kube-auth-proxy-6cd84dd4ff-z94r4\" (UID: \"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6\") " pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.158666 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.158626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-tls-certs\") pod \"kube-auth-proxy-6cd84dd4ff-z94r4\" (UID: \"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6\") " pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.158666 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.158672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4pdn\" (UniqueName: \"kubernetes.io/projected/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-kube-api-access-p4pdn\") pod \"kube-auth-proxy-6cd84dd4ff-z94r4\" (UID: \"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6\") " pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.158906 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.158693    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-tmp\") pod \"kube-auth-proxy-6cd84dd4ff-z94r4\" (UID: \"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6\") " pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.158906 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:51:24.158795    2581 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 18 18:51:24.158906 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:51:24.158872    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-tls-certs podName:fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6 nodeName:}" failed. No retries permitted until 2026-09-18 18:51:24.658854954 +0000 UTC m=+420.877862510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-tls-certs") pod "kube-auth-proxy-6cd84dd4ff-z94r4" (UID: "fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6") : secret "kube-auth-proxy-tls" not found
Sep 18 18:51:24.161166 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.161129    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-tmp\") pod \"kube-auth-proxy-6cd84dd4ff-z94r4\" (UID: \"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6\") " pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.188210 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.188180    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 18:51:24.201891 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.201850    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 18:51:24.209247 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.209212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4pdn\" (UniqueName: \"kubernetes.io/projected/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-kube-api-access-p4pdn\") pod \"kube-auth-proxy-6cd84dd4ff-z94r4\" (UID: \"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6\") " pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.661655 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.661604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-tls-certs\") pod \"kube-auth-proxy-6cd84dd4ff-z94r4\" (UID: \"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6\") " pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.664237 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.664200    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6-tls-certs\") pod \"kube-auth-proxy-6cd84dd4ff-z94r4\" (UID: \"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6\") " pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:24.929356 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.929312    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-v2zvp\""
Sep 18 18:51:24.937789 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:24.937754    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"
Sep 18 18:51:25.223170 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.222978    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4"]
Sep 18 18:51:25.225914 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:51:25.225885    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5e7fe7_96fa_44fb_bb5d_9bc4a449b3f6.slice/crio-7bd3aa0fcee214675dcec9c620701161aa43b31daa2f5fe74b3003eceb1d76b7 WatchSource:0}: Error finding container 7bd3aa0fcee214675dcec9c620701161aa43b31daa2f5fe74b3003eceb1d76b7: Status 404 returned error can't find the container with id 7bd3aa0fcee214675dcec9c620701161aa43b31daa2f5fe74b3003eceb1d76b7
Sep 18 18:51:25.484170 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.484086    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"]
Sep 18 18:51:25.487907 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.487877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.490935 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.490908    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 18:51:25.491222 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.491205    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 18:51:25.491300 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.491253    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-knzqd\""
Sep 18 18:51:25.491641 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.491627    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:51:25.491718 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.491668    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 18:51:25.512663 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.512622    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"]
Sep 18 18:51:25.567758 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.567715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.567758 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.567763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxqzh\" (UniqueName: \"kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-kube-api-access-kxqzh\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.568004 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.567785    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.568004 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.567815    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6585b6a-24b3-430a-b998-5c9b844bb279-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.568004 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.567831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.568004 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.567883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.568004 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.567897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.668998 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.668906    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.668998 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.668968    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqzh\" (UniqueName: \"kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-kube-api-access-kxqzh\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.668998 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.668997    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.669288 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.669037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6585b6a-24b3-430a-b998-5c9b844bb279-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.669288 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.669244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.669391 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.669299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.669391 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.669326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.669851 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.669822    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.672651 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.672618    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6585b6a-24b3-430a-b998-5c9b844bb279-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.672828 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.672726    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.672997 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.672972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.674653 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.674623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.678886 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.678849    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxqzh\" (UniqueName: \"kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-kube-api-access-kxqzh\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.679416 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.679384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-z26w7\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.798755 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.798631    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:25.973430 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:25.973383    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4" event={"ID":"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6","Type":"ContainerStarted","Data":"7bd3aa0fcee214675dcec9c620701161aa43b31daa2f5fe74b3003eceb1d76b7"}
Sep 18 18:51:26.097067 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:26.096732    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"]
Sep 18 18:51:26.099292 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:51:26.099251    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6585b6a_24b3_430a_b998_5c9b844bb279.slice/crio-b8ee1ee321404951450c9c5828279e25bc893d4e5e6a63d4910c02ec968af716 WatchSource:0}: Error finding container b8ee1ee321404951450c9c5828279e25bc893d4e5e6a63d4910c02ec968af716: Status 404 returned error can't find the container with id b8ee1ee321404951450c9c5828279e25bc893d4e5e6a63d4910c02ec968af716
Sep 18 18:51:26.982435 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:26.982381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7" event={"ID":"e6585b6a-24b3-430a-b998-5c9b844bb279","Type":"ContainerStarted","Data":"b8ee1ee321404951450c9c5828279e25bc893d4e5e6a63d4910c02ec968af716"}
Sep 18 18:51:27.834724 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:27.834678    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 18:51:27.994274 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:27.994233    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4" event={"ID":"fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6","Type":"ContainerStarted","Data":"e1eedc41b7b824ffd8bf97d68c9a95a89f018ca352c9532a51389dac46a875b7"}
Sep 18 18:51:28.029893 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:28.029804    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6cd84dd4ff-z94r4" podStartSLOduration=2.428837175 podStartE2EDuration="5.029781618s" totalImagesPullingTime="2.600944443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:28.021464728 +0000 UTC m=+424.240472320" watchObservedRunningTime="2026-09-18 18:51:28.029781618 +0000 UTC m=+424.248789198"
Sep 18 18:51:29.657653 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:29.657616    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:51:29.658087 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:29.657696    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:51:30.004976 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:30.004941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7" event={"ID":"e6585b6a-24b3-430a-b998-5c9b844bb279","Type":"ContainerStarted","Data":"284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26"}
Sep 18 18:51:30.005172 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:30.005000    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:30.007115 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:30.007084    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-z26w7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:51:30.007301 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:30.007146    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7" podUID="e6585b6a-24b3-430a-b998-5c9b844bb279" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:51:30.031781 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:30.031722    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7" podStartSLOduration=1.476557293 podStartE2EDuration="5.031679433s" totalImagesPullingTime="3.55512214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:30.030266175 +0000 UTC m=+426.249273753" watchObservedRunningTime="2026-09-18 18:51:30.031679433 +0000 UTC m=+426.250687011"
Sep 18 18:51:31.010310 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:31.010282    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:43.904798 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:43.904760    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"]
Sep 18 18:51:43.914622 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:43.914591    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:51:43.918131 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:43.918107    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 18 18:51:43.918369 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:43.918355    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5wlp9\""
Sep 18 18:51:43.928929 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:43.928899    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"]
Sep 18 18:51:43.934640 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:43.934608    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"]
Sep 18 18:51:43.937890 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:43.937863    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:51:43.940617 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:43.940585    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-qj78p\""
Sep 18 18:51:43.940798 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:43.940750    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 18 18:51:43.948955 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:43.948927    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"]
Sep 18 18:51:44.025119 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.025080    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4q6v\" (UniqueName: \"kubernetes.io/projected/6b51048b-0988-46e6-aec0-08e4061ad991-kube-api-access-v4q6v\") pod \"kserve-controller-manager-d6f4b65b8-h59gr\" (UID: \"6b51048b-0988-46e6-aec0-08e4061ad991\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:51:44.025119 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.025120    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5p6x\" (UniqueName: \"kubernetes.io/projected/680220d0-27f4-4b0c-9693-c685c08a1b67-kube-api-access-v5p6x\") pod \"llmisvc-controller-manager-777c6bd7d5-bb74d\" (UID: \"680220d0-27f4-4b0c-9693-c685c08a1b67\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:51:44.025392 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.025211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b51048b-0988-46e6-aec0-08e4061ad991-cert\") pod \"kserve-controller-manager-d6f4b65b8-h59gr\" (UID: \"6b51048b-0988-46e6-aec0-08e4061ad991\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:51:44.025392 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.025265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/680220d0-27f4-4b0c-9693-c685c08a1b67-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bb74d\" (UID: \"680220d0-27f4-4b0c-9693-c685c08a1b67\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:51:44.125984 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.125937    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b51048b-0988-46e6-aec0-08e4061ad991-cert\") pod \"kserve-controller-manager-d6f4b65b8-h59gr\" (UID: \"6b51048b-0988-46e6-aec0-08e4061ad991\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:51:44.125984 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.125983    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/680220d0-27f4-4b0c-9693-c685c08a1b67-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bb74d\" (UID: \"680220d0-27f4-4b0c-9693-c685c08a1b67\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:51:44.126194 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.126022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4q6v\" (UniqueName: \"kubernetes.io/projected/6b51048b-0988-46e6-aec0-08e4061ad991-kube-api-access-v4q6v\") pod \"kserve-controller-manager-d6f4b65b8-h59gr\" (UID: \"6b51048b-0988-46e6-aec0-08e4061ad991\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:51:44.126194 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.126040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5p6x\" (UniqueName: \"kubernetes.io/projected/680220d0-27f4-4b0c-9693-c685c08a1b67-kube-api-access-v5p6x\") pod \"llmisvc-controller-manager-777c6bd7d5-bb74d\" (UID: \"680220d0-27f4-4b0c-9693-c685c08a1b67\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:51:44.126194 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:51:44.126103    2581 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 18 18:51:44.126194 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:51:44.126175    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b51048b-0988-46e6-aec0-08e4061ad991-cert podName:6b51048b-0988-46e6-aec0-08e4061ad991 nodeName:}" failed. No retries permitted until 2026-09-18 18:51:44.626159493 +0000 UTC m=+440.845167049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6b51048b-0988-46e6-aec0-08e4061ad991-cert") pod "kserve-controller-manager-d6f4b65b8-h59gr" (UID: "6b51048b-0988-46e6-aec0-08e4061ad991") : secret "kserve-webhook-server-cert" not found
Sep 18 18:51:44.126322 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:51:44.126219    2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 18 18:51:44.126322 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:51:44.126281    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/680220d0-27f4-4b0c-9693-c685c08a1b67-cert podName:680220d0-27f4-4b0c-9693-c685c08a1b67 nodeName:}" failed. No retries permitted until 2026-09-18 18:51:44.626264322 +0000 UTC m=+440.845271878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/680220d0-27f4-4b0c-9693-c685c08a1b67-cert") pod "llmisvc-controller-manager-777c6bd7d5-bb74d" (UID: "680220d0-27f4-4b0c-9693-c685c08a1b67") : secret "llmisvc-webhook-server-cert" not found
Sep 18 18:51:44.141777 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.141739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4q6v\" (UniqueName: \"kubernetes.io/projected/6b51048b-0988-46e6-aec0-08e4061ad991-kube-api-access-v4q6v\") pod \"kserve-controller-manager-d6f4b65b8-h59gr\" (UID: \"6b51048b-0988-46e6-aec0-08e4061ad991\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:51:44.145204 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.145172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5p6x\" (UniqueName: \"kubernetes.io/projected/680220d0-27f4-4b0c-9693-c685c08a1b67-kube-api-access-v5p6x\") pod \"llmisvc-controller-manager-777c6bd7d5-bb74d\" (UID: \"680220d0-27f4-4b0c-9693-c685c08a1b67\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:51:44.631785 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.631743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b51048b-0988-46e6-aec0-08e4061ad991-cert\") pod \"kserve-controller-manager-d6f4b65b8-h59gr\" (UID: \"6b51048b-0988-46e6-aec0-08e4061ad991\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:51:44.631785 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.631791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/680220d0-27f4-4b0c-9693-c685c08a1b67-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bb74d\" (UID: \"680220d0-27f4-4b0c-9693-c685c08a1b67\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:51:44.634443 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.634422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b51048b-0988-46e6-aec0-08e4061ad991-cert\") pod \"kserve-controller-manager-d6f4b65b8-h59gr\" (UID: \"6b51048b-0988-46e6-aec0-08e4061ad991\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:51:44.634534 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.634422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/680220d0-27f4-4b0c-9693-c685c08a1b67-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bb74d\" (UID: \"680220d0-27f4-4b0c-9693-c685c08a1b67\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:51:44.825661 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.825618    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:51:44.850074 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:44.849989    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:51:45.124677 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:45.124650    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"]
Sep 18 18:51:45.127478 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:51:45.127443    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b51048b_0988_46e6_aec0_08e4061ad991.slice/crio-9b8448796df8ffaf0872f9a662fc23dab7fa30a7e38d95aeed9350ddb096aaf6 WatchSource:0}: Error finding container 9b8448796df8ffaf0872f9a662fc23dab7fa30a7e38d95aeed9350ddb096aaf6: Status 404 returned error can't find the container with id 9b8448796df8ffaf0872f9a662fc23dab7fa30a7e38d95aeed9350ddb096aaf6
Sep 18 18:51:45.144967 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:45.144940    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"]
Sep 18 18:51:45.147359 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:51:45.147315    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod680220d0_27f4_4b0c_9693_c685c08a1b67.slice/crio-c3e52c6f2f7d874fc36ac3deb43aeeef69025924ed70aec9b8b590bbf6807392 WatchSource:0}: Error finding container c3e52c6f2f7d874fc36ac3deb43aeeef69025924ed70aec9b8b590bbf6807392: Status 404 returned error can't find the container with id c3e52c6f2f7d874fc36ac3deb43aeeef69025924ed70aec9b8b590bbf6807392
Sep 18 18:51:46.080188 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:46.080137    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d" event={"ID":"680220d0-27f4-4b0c-9693-c685c08a1b67","Type":"ContainerStarted","Data":"c3e52c6f2f7d874fc36ac3deb43aeeef69025924ed70aec9b8b590bbf6807392"}
Sep 18 18:51:46.081509 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:46.081476    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr" event={"ID":"6b51048b-0988-46e6-aec0-08e4061ad991","Type":"ContainerStarted","Data":"9b8448796df8ffaf0872f9a662fc23dab7fa30a7e38d95aeed9350ddb096aaf6"}
Sep 18 18:51:49.096893 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:49.096848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d" event={"ID":"680220d0-27f4-4b0c-9693-c685c08a1b67","Type":"ContainerStarted","Data":"9fa2e0ead009cffb630211cba694aa74e0ae1d31d6ad163837fbe9b4390248ff"}
Sep 18 18:51:49.097351 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:49.096927    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:51:49.098459 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:49.098432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr" event={"ID":"6b51048b-0988-46e6-aec0-08e4061ad991","Type":"ContainerStarted","Data":"d144140b3a426b419498ef4076cb61fecfea8770199491a32252013faea95411"}
Sep 18 18:51:49.098594 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:49.098520    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:51:49.122450 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:49.122394    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d" podStartSLOduration=3.104736997 podStartE2EDuration="6.122375541s" totalImagesPullingTime="3.017638544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:49.121422496 +0000 UTC m=+445.340430073" watchObservedRunningTime="2026-09-18 18:51:49.122375541 +0000 UTC m=+445.341383143"
Sep 18 18:51:50.098453 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.098388    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr" podStartSLOduration=4.064762432 podStartE2EDuration="7.098370159s" totalImagesPullingTime="3.033607727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:49.149995938 +0000 UTC m=+445.369003516" watchObservedRunningTime="2026-09-18 18:51:50.098370159 +0000 UTC m=+446.317377738"
Sep 18 18:51:50.098898 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.098777    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"]
Sep 18 18:51:50.099042 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.099008    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7" podUID="e6585b6a-24b3-430a-b998-5c9b844bb279" containerName="discovery" containerID="cri-o://284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26" gracePeriod=30
Sep 18 18:51:50.354257 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.354162    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:50.485197 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.485157    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-ca-configmap\") pod \"e6585b6a-24b3-430a-b998-5c9b844bb279\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") "
Sep 18 18:51:50.485197 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.485194    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxqzh\" (UniqueName: \"kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-kube-api-access-kxqzh\") pod \"e6585b6a-24b3-430a-b998-5c9b844bb279\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") "
Sep 18 18:51:50.485457 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.485230    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-token\") pod \"e6585b6a-24b3-430a-b998-5c9b844bb279\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") "
Sep 18 18:51:50.485457 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.485266    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6585b6a-24b3-430a-b998-5c9b844bb279-local-certs\") pod \"e6585b6a-24b3-430a-b998-5c9b844bb279\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") "
Sep 18 18:51:50.485457 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.485285    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-kubeconfig\") pod \"e6585b6a-24b3-430a-b998-5c9b844bb279\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") "
Sep 18 18:51:50.485457 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.485318    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-dns-cert\") pod \"e6585b6a-24b3-430a-b998-5c9b844bb279\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") "
Sep 18 18:51:50.485457 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.485383    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-cacerts\") pod \"e6585b6a-24b3-430a-b998-5c9b844bb279\" (UID: \"e6585b6a-24b3-430a-b998-5c9b844bb279\") "
Sep 18 18:51:50.485728 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.485679    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e6585b6a-24b3-430a-b998-5c9b844bb279" (UID: "e6585b6a-24b3-430a-b998-5c9b844bb279"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:51:50.488201 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.488157    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6585b6a-24b3-430a-b998-5c9b844bb279-local-certs" (OuterVolumeSpecName: "local-certs") pod "e6585b6a-24b3-430a-b998-5c9b844bb279" (UID: "e6585b6a-24b3-430a-b998-5c9b844bb279"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:51:50.488201 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.488157    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e6585b6a-24b3-430a-b998-5c9b844bb279" (UID: "e6585b6a-24b3-430a-b998-5c9b844bb279"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:51:50.488417 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.488398    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-kube-api-access-kxqzh" (OuterVolumeSpecName: "kube-api-access-kxqzh") pod "e6585b6a-24b3-430a-b998-5c9b844bb279" (UID: "e6585b6a-24b3-430a-b998-5c9b844bb279"). InnerVolumeSpecName "kube-api-access-kxqzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:51:50.488485 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.488415    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e6585b6a-24b3-430a-b998-5c9b844bb279" (UID: "e6585b6a-24b3-430a-b998-5c9b844bb279"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:51:50.488485 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.488451    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-token" (OuterVolumeSpecName: "istio-token") pod "e6585b6a-24b3-430a-b998-5c9b844bb279" (UID: "e6585b6a-24b3-430a-b998-5c9b844bb279"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:51:50.488485 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.488476    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-cacerts" (OuterVolumeSpecName: "cacerts") pod "e6585b6a-24b3-430a-b998-5c9b844bb279" (UID: "e6585b6a-24b3-430a-b998-5c9b844bb279"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:51:50.586947 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.586907    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:51:50.586947 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.586939    2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6585b6a-24b3-430a-b998-5c9b844bb279-local-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:51:50.586947 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.586947    2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-kubeconfig\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:51:50.586947 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.586956    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-dns-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:51:50.587223 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.586966    2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6585b6a-24b3-430a-b998-5c9b844bb279-cacerts\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:51:50.587223 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.586974    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6585b6a-24b3-430a-b998-5c9b844bb279-istio-csr-ca-configmap\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:51:50.587223 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:50.586984    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxqzh\" (UniqueName: \"kubernetes.io/projected/e6585b6a-24b3-430a-b998-5c9b844bb279-kube-api-access-kxqzh\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:51:51.110813 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:51.110769    2581 generic.go:356] "Generic (PLEG): container finished" podID="e6585b6a-24b3-430a-b998-5c9b844bb279" containerID="284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26" exitCode=0
Sep 18 18:51:51.111301 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:51.110827    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"
Sep 18 18:51:51.111301 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:51.110826    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7" event={"ID":"e6585b6a-24b3-430a-b998-5c9b844bb279","Type":"ContainerDied","Data":"284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26"}
Sep 18 18:51:51.111301 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:51.110883    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7" event={"ID":"e6585b6a-24b3-430a-b998-5c9b844bb279","Type":"ContainerDied","Data":"b8ee1ee321404951450c9c5828279e25bc893d4e5e6a63d4910c02ec968af716"}
Sep 18 18:51:51.111301 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:51.110903    2581 scope.go:117] "RemoveContainer" containerID="284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26"
Sep 18 18:51:51.120298 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:51.120280    2581 scope.go:117] "RemoveContainer" containerID="284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26"
Sep 18 18:51:51.120639 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:51:51.120617    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26\": container with ID starting with 284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26 not found: ID does not exist" containerID="284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26"
Sep 18 18:51:51.120770 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:51.120646    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26"} err="failed to get container status \"284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26\": rpc error: code = NotFound desc = could not find container \"284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26\": container with ID starting with 284d44adce7917d163f0fdb1555a0a0ad143b97ecb28d06274000870f71eeb26 not found: ID does not exist"
Sep 18 18:51:51.133957 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:51.133919    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"]
Sep 18 18:51:51.142476 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:51.138258    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z26w7"]
Sep 18 18:51:52.312485 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:51:52.312441    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6585b6a-24b3-430a-b998-5c9b844bb279" path="/var/lib/kubelet/pods/e6585b6a-24b3-430a-b998-5c9b844bb279/volumes"
Sep 18 18:52:20.107012 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:20.106976    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb74d"
Sep 18 18:52:20.110510 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:20.110479    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h59gr"
Sep 18 18:52:40.260069 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.260029    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"]
Sep 18 18:52:40.260487 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.260429    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6585b6a-24b3-430a-b998-5c9b844bb279" containerName="discovery"
Sep 18 18:52:40.260487 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.260444    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6585b6a-24b3-430a-b998-5c9b844bb279" containerName="discovery"
Sep 18 18:52:40.260568 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.260540    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6585b6a-24b3-430a-b998-5c9b844bb279" containerName="discovery"
Sep 18 18:52:40.267466 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.267438    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.272259 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.272231    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 18:52:40.272393 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.272238    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mxnzd\""
Sep 18 18:52:40.273422 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.273386    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 18:52:40.276035 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.276009    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"]
Sep 18 18:52:40.350180 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.350130    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"]
Sep 18 18:52:40.353628 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.353606    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.362080 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.362052    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"]
Sep 18 18:52:40.384457 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.384418    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.384659 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.384466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.384659 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.384536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.384659 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.384586    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q768l\" (UniqueName: \"kubernetes.io/projected/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-kube-api-access-q768l\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.384659 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.384633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54zlr\" (UniqueName: \"kubernetes.io/projected/c98b8afc-e462-4444-9659-e5a4979f7176-kube-api-access-54zlr\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.384882 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.384688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.453567 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.453532    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"]
Sep 18 18:52:40.457083 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.457060    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.466464 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.466435    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"]
Sep 18 18:52:40.485866 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.485820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.485866 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.485866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.486102 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.485889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.486102 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.485925    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.486102 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.485978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn7r8\" (UniqueName: \"kubernetes.io/projected/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-kube-api-access-rn7r8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.486246 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.486119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q768l\" (UniqueName: \"kubernetes.io/projected/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-kube-api-access-q768l\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.486246 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.486183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54zlr\" (UniqueName: \"kubernetes.io/projected/c98b8afc-e462-4444-9659-e5a4979f7176-kube-api-access-54zlr\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.486375 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.486230    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.486375 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.486286    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.486375 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.486290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.486375 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.486329    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.486375 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.486334    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.486558 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.486469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.495574 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.495543    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54zlr\" (UniqueName: \"kubernetes.io/projected/c98b8afc-e462-4444-9659-e5a4979f7176-kube-api-access-54zlr\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.495574 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.495564    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q768l\" (UniqueName: \"kubernetes.io/projected/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-kube-api-access-q768l\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.551101 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.551005    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"]
Sep 18 18:52:40.554844 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.554821    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.570380 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.570347    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"]
Sep 18 18:52:40.577491 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.577463    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:40.586935 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.586904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.587061 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.586953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.587061 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.587008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn7r8\" (UniqueName: \"kubernetes.io/projected/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-kube-api-access-rn7r8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.587061 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.587043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj6mn\" (UniqueName: \"kubernetes.io/projected/150c0a60-6790-46d2-99d1-8bf3dea120f4-kube-api-access-gj6mn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.587212 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.587130    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.587212 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.587205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.587318 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.587298    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.587448 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.587431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.597099 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.597064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn7r8\" (UniqueName: \"kubernetes.io/projected/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-kube-api-access-rn7r8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.663269 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.663233    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:40.688622 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.688589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.688839 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.688656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.688839 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.688745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj6mn\" (UniqueName: \"kubernetes.io/projected/150c0a60-6790-46d2-99d1-8bf3dea120f4-kube-api-access-gj6mn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.689080 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.689059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.689133 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.689096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.701527 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.701496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj6mn\" (UniqueName: \"kubernetes.io/projected/150c0a60-6790-46d2-99d1-8bf3dea120f4-kube-api-access-gj6mn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.766072 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.766031    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:40.860214 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.860184    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"]
Sep 18 18:52:40.862389 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:52:40.862366    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98b8afc_e462_4444_9659_e5a4979f7176.slice/crio-d14c976234ce33be595e1d8f5ce65fad0804a2966e8039babb55da6a5f53dc85 WatchSource:0}: Error finding container d14c976234ce33be595e1d8f5ce65fad0804a2966e8039babb55da6a5f53dc85: Status 404 returned error can't find the container with id d14c976234ce33be595e1d8f5ce65fad0804a2966e8039babb55da6a5f53dc85
Sep 18 18:52:40.863857 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.863840    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:40.949957 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:40.949780    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"]
Sep 18 18:52:40.952233 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:52:40.952196    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebedb5cd_ff38_4ff0_9aa6_7c4fb3f23855.slice/crio-1317dada49ef1b15b8db3f341424839b4242311d86c4021ae1c6a860e5202060 WatchSource:0}: Error finding container 1317dada49ef1b15b8db3f341424839b4242311d86c4021ae1c6a860e5202060: Status 404 returned error can't find the container with id 1317dada49ef1b15b8db3f341424839b4242311d86c4021ae1c6a860e5202060
Sep 18 18:52:41.049898 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.049859    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"]
Sep 18 18:52:41.055497 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:52:41.055464    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9d51eb_3b7c_46e3_a394_aa3899104e8f.slice/crio-5aa7887ffa5c6e4bcbd43e0d5f27c1d763a00881074c91aaf7e439962202f957 WatchSource:0}: Error finding container 5aa7887ffa5c6e4bcbd43e0d5f27c1d763a00881074c91aaf7e439962202f957: Status 404 returned error can't find the container with id 5aa7887ffa5c6e4bcbd43e0d5f27c1d763a00881074c91aaf7e439962202f957
Sep 18 18:52:41.140941 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.140918    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"]
Sep 18 18:52:41.218816 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:52:41.218782    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150c0a60_6790_46d2_99d1_8bf3dea120f4.slice/crio-6c240e3b8b60185cf76a96093531e4108df2fe749bf605a9a3b7b7891988dee3 WatchSource:0}: Error finding container 6c240e3b8b60185cf76a96093531e4108df2fe749bf605a9a3b7b7891988dee3: Status 404 returned error can't find the container with id 6c240e3b8b60185cf76a96093531e4108df2fe749bf605a9a3b7b7891988dee3
Sep 18 18:52:41.329401 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.329368    2581 generic.go:356] "Generic (PLEG): container finished" podID="9d9d51eb-3b7c-46e3-a394-aa3899104e8f" containerID="db1c55d85ef3e4b183d292b1fd71d99b2204a81ff95b49a6905ba658a0ca90c6" exitCode=0
Sep 18 18:52:41.329779 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.329447    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd" event={"ID":"9d9d51eb-3b7c-46e3-a394-aa3899104e8f","Type":"ContainerDied","Data":"db1c55d85ef3e4b183d292b1fd71d99b2204a81ff95b49a6905ba658a0ca90c6"}
Sep 18 18:52:41.329779 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.329507    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd" event={"ID":"9d9d51eb-3b7c-46e3-a394-aa3899104e8f","Type":"ContainerStarted","Data":"5aa7887ffa5c6e4bcbd43e0d5f27c1d763a00881074c91aaf7e439962202f957"}
Sep 18 18:52:41.331187 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.331157    2581 generic.go:356] "Generic (PLEG): container finished" podID="ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" containerID="1af2da668aa02aa29a5a0cf07776043ed786f20268c147f63711f3497fc97bbc" exitCode=0
Sep 18 18:52:41.331300 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.331246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg" event={"ID":"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855","Type":"ContainerDied","Data":"1af2da668aa02aa29a5a0cf07776043ed786f20268c147f63711f3497fc97bbc"}
Sep 18 18:52:41.331300 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.331287    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg" event={"ID":"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855","Type":"ContainerStarted","Data":"1317dada49ef1b15b8db3f341424839b4242311d86c4021ae1c6a860e5202060"}
Sep 18 18:52:41.332833 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.332815    2581 generic.go:356] "Generic (PLEG): container finished" podID="c98b8afc-e462-4444-9659-e5a4979f7176" containerID="1aa918919f6696c8f54775caa4f7b8a5356949b37aa699564ca30d9a3cd168fb" exitCode=0
Sep 18 18:52:41.332916 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.332898    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm" event={"ID":"c98b8afc-e462-4444-9659-e5a4979f7176","Type":"ContainerDied","Data":"1aa918919f6696c8f54775caa4f7b8a5356949b37aa699564ca30d9a3cd168fb"}
Sep 18 18:52:41.332963 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.332938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm" event={"ID":"c98b8afc-e462-4444-9659-e5a4979f7176","Type":"ContainerStarted","Data":"d14c976234ce33be595e1d8f5ce65fad0804a2966e8039babb55da6a5f53dc85"}
Sep 18 18:52:41.334683 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.334580    2581 generic.go:356] "Generic (PLEG): container finished" podID="150c0a60-6790-46d2-99d1-8bf3dea120f4" containerID="4e69beb91b4d67bc74b48bd86555116b44912ae3a015c95fda8899da48ead504" exitCode=0
Sep 18 18:52:41.334683 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.334655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk" event={"ID":"150c0a60-6790-46d2-99d1-8bf3dea120f4","Type":"ContainerDied","Data":"4e69beb91b4d67bc74b48bd86555116b44912ae3a015c95fda8899da48ead504"}
Sep 18 18:52:41.334819 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:41.334688    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk" event={"ID":"150c0a60-6790-46d2-99d1-8bf3dea120f4","Type":"ContainerStarted","Data":"6c240e3b8b60185cf76a96093531e4108df2fe749bf605a9a3b7b7891988dee3"}
Sep 18 18:52:42.342568 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:42.342539    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm" event={"ID":"c98b8afc-e462-4444-9659-e5a4979f7176","Type":"ContainerStarted","Data":"6a38184d8e106ddf3de0cc76d0831234ce94345dd7b5265fb6db71a78a7a7f3a"}
Sep 18 18:52:43.349194 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:43.349157    2581 generic.go:356] "Generic (PLEG): container finished" podID="ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" containerID="1933be9316c23a6538856d2e1e96f6bec9b57028f93b70aa1335d5b392a169c5" exitCode=0
Sep 18 18:52:43.349676 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:43.349252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg" event={"ID":"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855","Type":"ContainerDied","Data":"1933be9316c23a6538856d2e1e96f6bec9b57028f93b70aa1335d5b392a169c5"}
Sep 18 18:52:43.351462 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:43.351435    2581 generic.go:356] "Generic (PLEG): container finished" podID="c98b8afc-e462-4444-9659-e5a4979f7176" containerID="6a38184d8e106ddf3de0cc76d0831234ce94345dd7b5265fb6db71a78a7a7f3a" exitCode=0
Sep 18 18:52:43.351550 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:43.351517    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm" event={"ID":"c98b8afc-e462-4444-9659-e5a4979f7176","Type":"ContainerDied","Data":"6a38184d8e106ddf3de0cc76d0831234ce94345dd7b5265fb6db71a78a7a7f3a"}
Sep 18 18:52:44.357860 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:44.357816    2581 generic.go:356] "Generic (PLEG): container finished" podID="ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" containerID="6719e6569a456e886fb3d8266d29f9a3a36efb91effd2259090d04534ab737ca" exitCode=0
Sep 18 18:52:44.358281 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:44.357907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg" event={"ID":"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855","Type":"ContainerDied","Data":"6719e6569a456e886fb3d8266d29f9a3a36efb91effd2259090d04534ab737ca"}
Sep 18 18:52:44.359993 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:44.359968    2581 generic.go:356] "Generic (PLEG): container finished" podID="c98b8afc-e462-4444-9659-e5a4979f7176" containerID="76cad7fe95a40556de3dbda7f579e8c2473effdaef86f99b43c1593e1b89891a" exitCode=0
Sep 18 18:52:44.360104 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:44.360057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm" event={"ID":"c98b8afc-e462-4444-9659-e5a4979f7176","Type":"ContainerDied","Data":"76cad7fe95a40556de3dbda7f579e8c2473effdaef86f99b43c1593e1b89891a"}
Sep 18 18:52:45.505573 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.505545    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:45.585972 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.585946    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:45.633309 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.633227    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q768l\" (UniqueName: \"kubernetes.io/projected/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-kube-api-access-q768l\") pod \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") "
Sep 18 18:52:45.633309 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.633262    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-bundle\") pod \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") "
Sep 18 18:52:45.633309 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.633285    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-util\") pod \"c98b8afc-e462-4444-9659-e5a4979f7176\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") "
Sep 18 18:52:45.633309 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.633304    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-bundle\") pod \"c98b8afc-e462-4444-9659-e5a4979f7176\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") "
Sep 18 18:52:45.633629 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.633338    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54zlr\" (UniqueName: \"kubernetes.io/projected/c98b8afc-e462-4444-9659-e5a4979f7176-kube-api-access-54zlr\") pod \"c98b8afc-e462-4444-9659-e5a4979f7176\" (UID: \"c98b8afc-e462-4444-9659-e5a4979f7176\") "
Sep 18 18:52:45.633629 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.633375    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-util\") pod \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\" (UID: \"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855\") "
Sep 18 18:52:45.633927 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.633883    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-bundle" (OuterVolumeSpecName: "bundle") pod "c98b8afc-e462-4444-9659-e5a4979f7176" (UID: "c98b8afc-e462-4444-9659-e5a4979f7176"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:52:45.634506 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.634478    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-bundle" (OuterVolumeSpecName: "bundle") pod "ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" (UID: "ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:52:45.635688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.635655    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-kube-api-access-q768l" (OuterVolumeSpecName: "kube-api-access-q768l") pod "ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" (UID: "ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855"). InnerVolumeSpecName "kube-api-access-q768l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:52:45.635894 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.635873    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c98b8afc-e462-4444-9659-e5a4979f7176-kube-api-access-54zlr" (OuterVolumeSpecName: "kube-api-access-54zlr") pod "c98b8afc-e462-4444-9659-e5a4979f7176" (UID: "c98b8afc-e462-4444-9659-e5a4979f7176"). InnerVolumeSpecName "kube-api-access-54zlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:52:45.640001 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.639954    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-util" (OuterVolumeSpecName: "util") pod "ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" (UID: "ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:52:45.640445 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.640425    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-util" (OuterVolumeSpecName: "util") pod "c98b8afc-e462-4444-9659-e5a4979f7176" (UID: "c98b8afc-e462-4444-9659-e5a4979f7176"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:52:45.734655 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.734621    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-util\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:45.734655 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.734650    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q768l\" (UniqueName: \"kubernetes.io/projected/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-kube-api-access-q768l\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:45.734655 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.734659    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855-bundle\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:45.734900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.734669    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-util\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:45.734900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.734676    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c98b8afc-e462-4444-9659-e5a4979f7176-bundle\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:45.734900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:45.734684    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54zlr\" (UniqueName: \"kubernetes.io/projected/c98b8afc-e462-4444-9659-e5a4979f7176-kube-api-access-54zlr\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:46.371558 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:46.371521    2581 generic.go:356] "Generic (PLEG): container finished" podID="9d9d51eb-3b7c-46e3-a394-aa3899104e8f" containerID="83184bbdfc1dde2afc37149e9745390eaee9d87fe49b7f456601435862cdac9a" exitCode=0
Sep 18 18:52:46.371748 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:46.371594    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd" event={"ID":"9d9d51eb-3b7c-46e3-a394-aa3899104e8f","Type":"ContainerDied","Data":"83184bbdfc1dde2afc37149e9745390eaee9d87fe49b7f456601435862cdac9a"}
Sep 18 18:52:46.373810 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:46.373786    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg" event={"ID":"ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855","Type":"ContainerDied","Data":"1317dada49ef1b15b8db3f341424839b4242311d86c4021ae1c6a860e5202060"}
Sep 18 18:52:46.373905 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:46.373828    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1317dada49ef1b15b8db3f341424839b4242311d86c4021ae1c6a860e5202060"
Sep 18 18:52:46.373905 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:46.373794    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afprxg"
Sep 18 18:52:46.375824 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:46.375802    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm"
Sep 18 18:52:46.375930 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:46.375793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w9jkm" event={"ID":"c98b8afc-e462-4444-9659-e5a4979f7176","Type":"ContainerDied","Data":"d14c976234ce33be595e1d8f5ce65fad0804a2966e8039babb55da6a5f53dc85"}
Sep 18 18:52:46.375998 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:46.375953    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d14c976234ce33be595e1d8f5ce65fad0804a2966e8039babb55da6a5f53dc85"
Sep 18 18:52:47.382053 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:47.382014    2581 generic.go:356] "Generic (PLEG): container finished" podID="150c0a60-6790-46d2-99d1-8bf3dea120f4" containerID="57c5e01a88d9fa47d130af281201974be75a291a71adf756e028ade4ae1a9f4a" exitCode=0
Sep 18 18:52:47.382470 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:47.382106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk" event={"ID":"150c0a60-6790-46d2-99d1-8bf3dea120f4","Type":"ContainerDied","Data":"57c5e01a88d9fa47d130af281201974be75a291a71adf756e028ade4ae1a9f4a"}
Sep 18 18:52:47.384305 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:47.384278    2581 generic.go:356] "Generic (PLEG): container finished" podID="9d9d51eb-3b7c-46e3-a394-aa3899104e8f" containerID="6abe6522b72fb0ee84c8f188f19b37adb0beeca7fb1f46fe3daebfb686c6ddab" exitCode=0
Sep 18 18:52:47.384390 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:47.384341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd" event={"ID":"9d9d51eb-3b7c-46e3-a394-aa3899104e8f","Type":"ContainerDied","Data":"6abe6522b72fb0ee84c8f188f19b37adb0beeca7fb1f46fe3daebfb686c6ddab"}
Sep 18 18:52:48.391903 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.391872    2581 generic.go:356] "Generic (PLEG): container finished" podID="150c0a60-6790-46d2-99d1-8bf3dea120f4" containerID="45343d8b3ac7508334ffeb46bf76550959df02d1270d1d2019c3472c24bd947e" exitCode=0
Sep 18 18:52:48.392282 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.391972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk" event={"ID":"150c0a60-6790-46d2-99d1-8bf3dea120f4","Type":"ContainerDied","Data":"45343d8b3ac7508334ffeb46bf76550959df02d1270d1d2019c3472c24bd947e"}
Sep 18 18:52:48.517916 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.517887    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:48.658104 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.658013    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-bundle\") pod \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") "
Sep 18 18:52:48.658104 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.658060    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-util\") pod \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") "
Sep 18 18:52:48.658104 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.658101    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn7r8\" (UniqueName: \"kubernetes.io/projected/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-kube-api-access-rn7r8\") pod \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\" (UID: \"9d9d51eb-3b7c-46e3-a394-aa3899104e8f\") "
Sep 18 18:52:48.658496 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.658461    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-bundle" (OuterVolumeSpecName: "bundle") pod "9d9d51eb-3b7c-46e3-a394-aa3899104e8f" (UID: "9d9d51eb-3b7c-46e3-a394-aa3899104e8f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:52:48.660356 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.660329    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-kube-api-access-rn7r8" (OuterVolumeSpecName: "kube-api-access-rn7r8") pod "9d9d51eb-3b7c-46e3-a394-aa3899104e8f" (UID: "9d9d51eb-3b7c-46e3-a394-aa3899104e8f"). InnerVolumeSpecName "kube-api-access-rn7r8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:52:48.662786 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.662732    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-util" (OuterVolumeSpecName: "util") pod "9d9d51eb-3b7c-46e3-a394-aa3899104e8f" (UID: "9d9d51eb-3b7c-46e3-a394-aa3899104e8f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:52:48.758781 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.758741    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-util\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:48.758781 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.758773    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn7r8\" (UniqueName: \"kubernetes.io/projected/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-kube-api-access-rn7r8\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:48.758781 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:48.758782    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d9d51eb-3b7c-46e3-a394-aa3899104e8f-bundle\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:49.398492 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.398462    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd"
Sep 18 18:52:49.398893 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.398498    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7kxpmd" event={"ID":"9d9d51eb-3b7c-46e3-a394-aa3899104e8f","Type":"ContainerDied","Data":"5aa7887ffa5c6e4bcbd43e0d5f27c1d763a00881074c91aaf7e439962202f957"}
Sep 18 18:52:49.398893 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.398544    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5aa7887ffa5c6e4bcbd43e0d5f27c1d763a00881074c91aaf7e439962202f957"
Sep 18 18:52:49.528016 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.527991    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:49.665302 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.665271    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gj6mn\" (UniqueName: \"kubernetes.io/projected/150c0a60-6790-46d2-99d1-8bf3dea120f4-kube-api-access-gj6mn\") pod \"150c0a60-6790-46d2-99d1-8bf3dea120f4\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") "
Sep 18 18:52:49.665470 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.665318    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-util\") pod \"150c0a60-6790-46d2-99d1-8bf3dea120f4\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") "
Sep 18 18:52:49.665470 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.665388    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-bundle\") pod \"150c0a60-6790-46d2-99d1-8bf3dea120f4\" (UID: \"150c0a60-6790-46d2-99d1-8bf3dea120f4\") "
Sep 18 18:52:49.665982 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.665955    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-bundle" (OuterVolumeSpecName: "bundle") pod "150c0a60-6790-46d2-99d1-8bf3dea120f4" (UID: "150c0a60-6790-46d2-99d1-8bf3dea120f4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:52:49.667480 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.667454    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/150c0a60-6790-46d2-99d1-8bf3dea120f4-kube-api-access-gj6mn" (OuterVolumeSpecName: "kube-api-access-gj6mn") pod "150c0a60-6790-46d2-99d1-8bf3dea120f4" (UID: "150c0a60-6790-46d2-99d1-8bf3dea120f4"). InnerVolumeSpecName "kube-api-access-gj6mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:52:49.669854 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.669808    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-util" (OuterVolumeSpecName: "util") pod "150c0a60-6790-46d2-99d1-8bf3dea120f4" (UID: "150c0a60-6790-46d2-99d1-8bf3dea120f4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:52:49.766297 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.766257    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-bundle\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:49.766297 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.766293    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj6mn\" (UniqueName: \"kubernetes.io/projected/150c0a60-6790-46d2-99d1-8bf3dea120f4-kube-api-access-gj6mn\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:49.766297 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:49.766304    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/150c0a60-6790-46d2-99d1-8bf3dea120f4-util\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:52:50.405132 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:50.405096    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk" event={"ID":"150c0a60-6790-46d2-99d1-8bf3dea120f4","Type":"ContainerDied","Data":"6c240e3b8b60185cf76a96093531e4108df2fe749bf605a9a3b7b7891988dee3"}
Sep 18 18:52:50.405132 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:50.405132    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2f8kk"
Sep 18 18:52:50.405689 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:50.405144    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c240e3b8b60185cf76a96093531e4108df2fe749bf605a9a3b7b7891988dee3"
Sep 18 18:52:58.094740 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.094686    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"]
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095004    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d9d51eb-3b7c-46e3-a394-aa3899104e8f" containerName="util"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095015    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d9d51eb-3b7c-46e3-a394-aa3899104e8f" containerName="util"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095024    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="150c0a60-6790-46d2-99d1-8bf3dea120f4" containerName="util"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095030    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="150c0a60-6790-46d2-99d1-8bf3dea120f4" containerName="util"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095035    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" containerName="pull"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095041    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" containerName="pull"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095048    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" containerName="util"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095053    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" containerName="util"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095062    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="150c0a60-6790-46d2-99d1-8bf3dea120f4" containerName="pull"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095067    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="150c0a60-6790-46d2-99d1-8bf3dea120f4" containerName="pull"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095076    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095082    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095087    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d9d51eb-3b7c-46e3-a394-aa3899104e8f" containerName="pull"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095092    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d9d51eb-3b7c-46e3-a394-aa3899104e8f" containerName="pull"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095097    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c98b8afc-e462-4444-9659-e5a4979f7176" containerName="pull"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095102    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c98b8afc-e462-4444-9659-e5a4979f7176" containerName="pull"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095108    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c98b8afc-e462-4444-9659-e5a4979f7176" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095112    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c98b8afc-e462-4444-9659-e5a4979f7176" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095120    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c98b8afc-e462-4444-9659-e5a4979f7176" containerName="util"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095124    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c98b8afc-e462-4444-9659-e5a4979f7176" containerName="util"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095132    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d9d51eb-3b7c-46e3-a394-aa3899104e8f" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095138    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d9d51eb-3b7c-46e3-a394-aa3899104e8f" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095144    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="150c0a60-6790-46d2-99d1-8bf3dea120f4" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095148    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="150c0a60-6790-46d2-99d1-8bf3dea120f4" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095194    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d9d51eb-3b7c-46e3-a394-aa3899104e8f" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095203    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c98b8afc-e462-4444-9659-e5a4979f7176" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095209    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebedb5cd-ff38-4ff0-9aa6-7c4fb3f23855" containerName="extract"
Sep 18 18:52:58.095213 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.095216    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="150c0a60-6790-46d2-99d1-8bf3dea120f4" containerName="extract"
Sep 18 18:52:58.098469 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.098441    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:52:58.105097 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.104810    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-grlk2\""
Sep 18 18:52:58.105097 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.104966    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 18:52:58.105505 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.105486    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 18:52:58.134867 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.134834    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"]
Sep 18 18:52:58.234112 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.234069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68g8f\" (UniqueName: \"kubernetes.io/projected/d6e52851-7f1f-4915-9240-243faf6b0ced-kube-api-access-68g8f\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" (UID: \"d6e52851-7f1f-4915-9240-243faf6b0ced\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:52:58.234304 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.234139    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6e52851-7f1f-4915-9240-243faf6b0ced-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" (UID: \"d6e52851-7f1f-4915-9240-243faf6b0ced\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:52:58.335201 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.335153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68g8f\" (UniqueName: \"kubernetes.io/projected/d6e52851-7f1f-4915-9240-243faf6b0ced-kube-api-access-68g8f\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" (UID: \"d6e52851-7f1f-4915-9240-243faf6b0ced\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:52:58.335381 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.335224    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6e52851-7f1f-4915-9240-243faf6b0ced-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" (UID: \"d6e52851-7f1f-4915-9240-243faf6b0ced\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:52:58.335595 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.335573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6e52851-7f1f-4915-9240-243faf6b0ced-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" (UID: \"d6e52851-7f1f-4915-9240-243faf6b0ced\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:52:58.350228 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.350165    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68g8f\" (UniqueName: \"kubernetes.io/projected/d6e52851-7f1f-4915-9240-243faf6b0ced-kube-api-access-68g8f\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" (UID: \"d6e52851-7f1f-4915-9240-243faf6b0ced\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:52:58.408299 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.408259    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:52:58.702144 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:58.702120    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"]
Sep 18 18:52:58.703852 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:52:58.703826    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e52851_7f1f_4915_9240_243faf6b0ced.slice/crio-a566af678cc02d84e5a1ce53a4c129d71f4b68c423db470fb87981d79d21b5ad WatchSource:0}: Error finding container a566af678cc02d84e5a1ce53a4c129d71f4b68c423db470fb87981d79d21b5ad: Status 404 returned error can't find the container with id a566af678cc02d84e5a1ce53a4c129d71f4b68c423db470fb87981d79d21b5ad
Sep 18 18:52:59.448010 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:59.447973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc" event={"ID":"d6e52851-7f1f-4915-9240-243faf6b0ced","Type":"ContainerStarted","Data":"a566af678cc02d84e5a1ce53a4c129d71f4b68c423db470fb87981d79d21b5ad"}
Sep 18 18:52:59.946301 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:59.946261    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"]
Sep 18 18:52:59.950808 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:59.950785    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:52:59.956254 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:59.956226    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4w24w\""
Sep 18 18:52:59.979648 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:52:59.979608    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"]
Sep 18 18:53:00.049062 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:00.049018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4csdr\" (UniqueName: \"kubernetes.io/projected/5ca964b6-de47-4e24-8f39-e316edf35898-kube-api-access-4csdr\") pod \"limitador-operator-controller-manager-74495c69f8-smr4p\" (UID: \"5ca964b6-de47-4e24-8f39-e316edf35898\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:00.149897 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:00.149857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4csdr\" (UniqueName: \"kubernetes.io/projected/5ca964b6-de47-4e24-8f39-e316edf35898-kube-api-access-4csdr\") pod \"limitador-operator-controller-manager-74495c69f8-smr4p\" (UID: \"5ca964b6-de47-4e24-8f39-e316edf35898\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:00.183613 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:00.183571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4csdr\" (UniqueName: \"kubernetes.io/projected/5ca964b6-de47-4e24-8f39-e316edf35898-kube-api-access-4csdr\") pod \"limitador-operator-controller-manager-74495c69f8-smr4p\" (UID: \"5ca964b6-de47-4e24-8f39-e316edf35898\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:00.260925 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:00.260818    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:00.611480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:53:00.611417    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ca964b6_de47_4e24_8f39_e316edf35898.slice/crio-ef7e9edb0a725794e69879b4f5560cb28e55d07a6486f659ff0c1c3e33897fba WatchSource:0}: Error finding container ef7e9edb0a725794e69879b4f5560cb28e55d07a6486f659ff0c1c3e33897fba: Status 404 returned error can't find the container with id ef7e9edb0a725794e69879b4f5560cb28e55d07a6486f659ff0c1c3e33897fba
Sep 18 18:53:00.614242 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:00.614217    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"]
Sep 18 18:53:01.466960 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:01.466922    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p" event={"ID":"5ca964b6-de47-4e24-8f39-e316edf35898","Type":"ContainerStarted","Data":"ef7e9edb0a725794e69879b4f5560cb28e55d07a6486f659ff0c1c3e33897fba"}
Sep 18 18:53:02.406734 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:02.406676    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd"]
Sep 18 18:53:02.414090 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:02.414054    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd"
Sep 18 18:53:02.420526 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:02.420272    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 18 18:53:02.420526 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:02.420344    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5tv6s\""
Sep 18 18:53:02.465244 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:02.465130    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd"]
Sep 18 18:53:02.570239 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:02.570207    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggvs8\" (UniqueName: \"kubernetes.io/projected/80562742-a164-47ef-87d7-60f2b67ef4db-kube-api-access-ggvs8\") pod \"dns-operator-controller-manager-7b84cd7c9c-gdwhd\" (UID: \"80562742-a164-47ef-87d7-60f2b67ef4db\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd"
Sep 18 18:53:02.671274 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:02.671198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggvs8\" (UniqueName: \"kubernetes.io/projected/80562742-a164-47ef-87d7-60f2b67ef4db-kube-api-access-ggvs8\") pod \"dns-operator-controller-manager-7b84cd7c9c-gdwhd\" (UID: \"80562742-a164-47ef-87d7-60f2b67ef4db\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd"
Sep 18 18:53:02.688668 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:02.688442    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggvs8\" (UniqueName: \"kubernetes.io/projected/80562742-a164-47ef-87d7-60f2b67ef4db-kube-api-access-ggvs8\") pod \"dns-operator-controller-manager-7b84cd7c9c-gdwhd\" (UID: \"80562742-a164-47ef-87d7-60f2b67ef4db\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd"
Sep 18 18:53:02.729866 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:02.729716    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd"
Sep 18 18:53:03.816358 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:03.816329    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd"]
Sep 18 18:53:03.818370 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:53:03.818344    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80562742_a164_47ef_87d7_60f2b67ef4db.slice/crio-2977cf6421d55adf33932e1a002b60685f2150ba7d9808673dd40af0e38649ce WatchSource:0}: Error finding container 2977cf6421d55adf33932e1a002b60685f2150ba7d9808673dd40af0e38649ce: Status 404 returned error can't find the container with id 2977cf6421d55adf33932e1a002b60685f2150ba7d9808673dd40af0e38649ce
Sep 18 18:53:04.486781 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:04.486739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd" event={"ID":"80562742-a164-47ef-87d7-60f2b67ef4db","Type":"ContainerStarted","Data":"2977cf6421d55adf33932e1a002b60685f2150ba7d9808673dd40af0e38649ce"}
Sep 18 18:53:04.488828 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:04.488787    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p" event={"ID":"5ca964b6-de47-4e24-8f39-e316edf35898","Type":"ContainerStarted","Data":"189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc"}
Sep 18 18:53:04.488974 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:04.488923    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:04.523408 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:04.523351    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p" podStartSLOduration=2.573754711 podStartE2EDuration="5.523331877s" totalImagesPullingTime="2.949577166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:52:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:04.521206969 +0000 UTC m=+520.740214546" watchObservedRunningTime="2026-09-18 18:53:04.523331877 +0000 UTC m=+520.742339454"
Sep 18 18:53:05.496609 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.496515    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc" event={"ID":"d6e52851-7f1f-4915-9240-243faf6b0ced","Type":"ContainerStarted","Data":"e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13"}
Sep 18 18:53:05.497148 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.496634    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:05.509361 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.509324    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-xq5pf"]
Sep 18 18:53:05.513186 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.513158    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-xq5pf"
Sep 18 18:53:05.518196 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.518170    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vdt5k\""
Sep 18 18:53:05.549241 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.549206    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-xq5pf"]
Sep 18 18:53:05.580650 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.580588    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc" podStartSLOduration=0.874016451 podStartE2EDuration="7.580569332s" totalImagesPullingTime="6.706552881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:05.570147955 +0000 UTC m=+521.789155527" watchObservedRunningTime="2026-09-18 18:53:05.580569332 +0000 UTC m=+521.799576910"
Sep 18 18:53:05.597981 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.597948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stbfq\" (UniqueName: \"kubernetes.io/projected/b978db62-0178-43d7-8ec2-a001bcd2ce96-kube-api-access-stbfq\") pod \"authorino-operator-76b478bd78-xq5pf\" (UID: \"b978db62-0178-43d7-8ec2-a001bcd2ce96\") " pod="kuadrant-system/authorino-operator-76b478bd78-xq5pf"
Sep 18 18:53:05.698818 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.698783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stbfq\" (UniqueName: \"kubernetes.io/projected/b978db62-0178-43d7-8ec2-a001bcd2ce96-kube-api-access-stbfq\") pod \"authorino-operator-76b478bd78-xq5pf\" (UID: \"b978db62-0178-43d7-8ec2-a001bcd2ce96\") " pod="kuadrant-system/authorino-operator-76b478bd78-xq5pf"
Sep 18 18:53:05.710276 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.710248    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stbfq\" (UniqueName: \"kubernetes.io/projected/b978db62-0178-43d7-8ec2-a001bcd2ce96-kube-api-access-stbfq\") pod \"authorino-operator-76b478bd78-xq5pf\" (UID: \"b978db62-0178-43d7-8ec2-a001bcd2ce96\") " pod="kuadrant-system/authorino-operator-76b478bd78-xq5pf"
Sep 18 18:53:05.825733 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:05.825618    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-xq5pf"
Sep 18 18:53:06.195695 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:06.195664    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-xq5pf"]
Sep 18 18:53:06.203126 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:53:06.203089    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb978db62_0178_43d7_8ec2_a001bcd2ce96.slice/crio-b203441c0ac70d2ab9984b63812c4173168d50d8adb4a0df53ddeff0fbb8c680 WatchSource:0}: Error finding container b203441c0ac70d2ab9984b63812c4173168d50d8adb4a0df53ddeff0fbb8c680: Status 404 returned error can't find the container with id b203441c0ac70d2ab9984b63812c4173168d50d8adb4a0df53ddeff0fbb8c680
Sep 18 18:53:06.504477 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:06.504384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-xq5pf" event={"ID":"b978db62-0178-43d7-8ec2-a001bcd2ce96","Type":"ContainerStarted","Data":"b203441c0ac70d2ab9984b63812c4173168d50d8adb4a0df53ddeff0fbb8c680"}
Sep 18 18:53:07.512943 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:07.512902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd" event={"ID":"80562742-a164-47ef-87d7-60f2b67ef4db","Type":"ContainerStarted","Data":"82a990d53c030794c18bae727db1d08d621da796fec4641b171927913bf35b24"}
Sep 18 18:53:07.513374 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:07.512974    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd"
Sep 18 18:53:07.574660 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:07.574604    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd" podStartSLOduration=2.728378948 podStartE2EDuration="5.574585286s" totalImagesPullingTime="2.846206338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:07.572260239 +0000 UTC m=+523.791267816" watchObservedRunningTime="2026-09-18 18:53:07.574585286 +0000 UTC m=+523.793592917"
Sep 18 18:53:08.519630 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:08.519590    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-xq5pf" event={"ID":"b978db62-0178-43d7-8ec2-a001bcd2ce96","Type":"ContainerStarted","Data":"01e3cce264adc766431b4a9ebb61261038d7408147e72b30f639c2b2a6627773"}
Sep 18 18:53:08.572963 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:08.572905    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-xq5pf" podStartSLOduration=2.332965553 podStartE2EDuration="3.572886073s" totalImagesPullingTime="1.23992052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:08.57062604 +0000 UTC m=+524.789633617" watchObservedRunningTime="2026-09-18 18:53:08.572886073 +0000 UTC m=+524.791893651"
Sep 18 18:53:09.525044 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:09.525007    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-xq5pf"
Sep 18 18:53:15.499421 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:15.499329    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:16.507550 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:16.507511    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:18.522516 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:18.522485    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gdwhd"
Sep 18 18:53:20.516561 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.516526    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"]
Sep 18 18:53:20.517018 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.516751    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" containerName="manager" containerID="cri-o://e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13" gracePeriod=2
Sep 18 18:53:20.533118 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.533079    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-xq5pf"
Sep 18 18:53:20.541738 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:20.541676    2581 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d6e52851-7f1f-4915-9240-243faf6b0ced\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\": pods \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:20.545187 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.545155    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"]
Sep 18 18:53:20.546645 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:20.546608    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:20.588112 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.588081    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"]
Sep 18 18:53:20.588355 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.588330    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p" podUID="5ca964b6-de47-4e24-8f39-e316edf35898" containerName="manager" containerID="cri-o://189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc" gracePeriod=2
Sep 18 18:53:20.598197 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:20.598155    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:20.663479 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.663410    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"]
Sep 18 18:53:20.666245 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.666206    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"]
Sep 18 18:53:20.666782 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.666762    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ca964b6-de47-4e24-8f39-e316edf35898" containerName="manager"
Sep 18 18:53:20.666858 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.666785    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ca964b6-de47-4e24-8f39-e316edf35898" containerName="manager"
Sep 18 18:53:20.666858 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.666815    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" containerName="manager"
Sep 18 18:53:20.666858 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.666822    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" containerName="manager"
Sep 18 18:53:20.667013 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.666912    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ca964b6-de47-4e24-8f39-e316edf35898" containerName="manager"
Sep 18 18:53:20.667013 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.666924    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" containerName="manager"
Sep 18 18:53:20.670194 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.670172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:20.674011 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:20.673970    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:20.686946 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:20.686907    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-smr4p\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="5ca964b6-de47-4e24-8f39-e316edf35898" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:20.719900 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.719857    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl"]
Sep 18 18:53:20.723647 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.723622    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl"
Sep 18 18:53:20.768378 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.768295    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"]
Sep 18 18:53:20.807249 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.807211    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl"]
Sep 18 18:53:20.824473 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.824438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpsn6\" (UniqueName: \"kubernetes.io/projected/f9e990f7-49a2-4b1e-8889-862fe80e4106-kube-api-access-fpsn6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-f5lrn\" (UID: \"f9e990f7-49a2-4b1e-8889-862fe80e4106\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:20.824625 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.824486    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e990f7-49a2-4b1e-8889-862fe80e4106-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-f5lrn\" (UID: \"f9e990f7-49a2-4b1e-8889-862fe80e4106\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:20.824625 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.824517    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfrd8\" (UniqueName: \"kubernetes.io/projected/081fc10e-a177-4bbc-977e-41978480fb31-kube-api-access-pfrd8\") pod \"limitador-operator-controller-manager-74495c69f8-ddjfl\" (UID: \"081fc10e-a177-4bbc-977e-41978480fb31\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl"
Sep 18 18:53:20.868559 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.868529    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:20.870955 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.870924    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:20.893195 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:20.893155    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:20.897168 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:20.897141    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-smr4p\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="5ca964b6-de47-4e24-8f39-e316edf35898" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:20.925765 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.925722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpsn6\" (UniqueName: \"kubernetes.io/projected/f9e990f7-49a2-4b1e-8889-862fe80e4106-kube-api-access-fpsn6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-f5lrn\" (UID: \"f9e990f7-49a2-4b1e-8889-862fe80e4106\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:20.925765 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.925777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e990f7-49a2-4b1e-8889-862fe80e4106-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-f5lrn\" (UID: \"f9e990f7-49a2-4b1e-8889-862fe80e4106\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:20.926051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.925798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfrd8\" (UniqueName: \"kubernetes.io/projected/081fc10e-a177-4bbc-977e-41978480fb31-kube-api-access-pfrd8\") pod \"limitador-operator-controller-manager-74495c69f8-ddjfl\" (UID: \"081fc10e-a177-4bbc-977e-41978480fb31\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl"
Sep 18 18:53:20.926198 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.926176    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e990f7-49a2-4b1e-8889-862fe80e4106-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-f5lrn\" (UID: \"f9e990f7-49a2-4b1e-8889-862fe80e4106\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:20.952465 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.952427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfrd8\" (UniqueName: \"kubernetes.io/projected/081fc10e-a177-4bbc-977e-41978480fb31-kube-api-access-pfrd8\") pod \"limitador-operator-controller-manager-74495c69f8-ddjfl\" (UID: \"081fc10e-a177-4bbc-977e-41978480fb31\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl"
Sep 18 18:53:20.988993 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:20.988961    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpsn6\" (UniqueName: \"kubernetes.io/projected/f9e990f7-49a2-4b1e-8889-862fe80e4106-kube-api-access-fpsn6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-f5lrn\" (UID: \"f9e990f7-49a2-4b1e-8889-862fe80e4106\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:21.026943 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.026856    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4csdr\" (UniqueName: \"kubernetes.io/projected/5ca964b6-de47-4e24-8f39-e316edf35898-kube-api-access-4csdr\") pod \"5ca964b6-de47-4e24-8f39-e316edf35898\" (UID: \"5ca964b6-de47-4e24-8f39-e316edf35898\") "
Sep 18 18:53:21.026943 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.026914    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6e52851-7f1f-4915-9240-243faf6b0ced-extensions-socket-volume\") pod \"d6e52851-7f1f-4915-9240-243faf6b0ced\" (UID: \"d6e52851-7f1f-4915-9240-243faf6b0ced\") "
Sep 18 18:53:21.027112 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.026968    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68g8f\" (UniqueName: \"kubernetes.io/projected/d6e52851-7f1f-4915-9240-243faf6b0ced-kube-api-access-68g8f\") pod \"d6e52851-7f1f-4915-9240-243faf6b0ced\" (UID: \"d6e52851-7f1f-4915-9240-243faf6b0ced\") "
Sep 18 18:53:21.027353 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.027332    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6e52851-7f1f-4915-9240-243faf6b0ced-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d6e52851-7f1f-4915-9240-243faf6b0ced" (UID: "d6e52851-7f1f-4915-9240-243faf6b0ced"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:53:21.029378 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.029349    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ca964b6-de47-4e24-8f39-e316edf35898-kube-api-access-4csdr" (OuterVolumeSpecName: "kube-api-access-4csdr") pod "5ca964b6-de47-4e24-8f39-e316edf35898" (UID: "5ca964b6-de47-4e24-8f39-e316edf35898"). InnerVolumeSpecName "kube-api-access-4csdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:53:21.029378 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.029366    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e52851-7f1f-4915-9240-243faf6b0ced-kube-api-access-68g8f" (OuterVolumeSpecName: "kube-api-access-68g8f") pod "d6e52851-7f1f-4915-9240-243faf6b0ced" (UID: "d6e52851-7f1f-4915-9240-243faf6b0ced"). InnerVolumeSpecName "kube-api-access-68g8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:53:21.035085 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.035057    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl"
Sep 18 18:53:21.128276 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.128230    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6e52851-7f1f-4915-9240-243faf6b0ced-extensions-socket-volume\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:53:21.128276 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.128267    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68g8f\" (UniqueName: \"kubernetes.io/projected/d6e52851-7f1f-4915-9240-243faf6b0ced-kube-api-access-68g8f\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:53:21.128276 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.128276    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4csdr\" (UniqueName: \"kubernetes.io/projected/5ca964b6-de47-4e24-8f39-e316edf35898-kube-api-access-4csdr\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:53:21.282972 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.282875    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:21.328799 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.328774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl"]
Sep 18 18:53:21.331808 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:53:21.331774    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081fc10e_a177_4bbc_977e_41978480fb31.slice/crio-0f352ec5386736f571854c325204e8e2c984c805950d5dba5be76d4a0fdb1a0d WatchSource:0}: Error finding container 0f352ec5386736f571854c325204e8e2c984c805950d5dba5be76d4a0fdb1a0d: Status 404 returned error can't find the container with id 0f352ec5386736f571854c325204e8e2c984c805950d5dba5be76d4a0fdb1a0d
Sep 18 18:53:21.589145 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.589105    2581 generic.go:356] "Generic (PLEG): container finished" podID="d6e52851-7f1f-4915-9240-243faf6b0ced" containerID="e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13" exitCode=0
Sep 18 18:53:21.589624 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.589180    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:21.589624 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.589204    2581 scope.go:117] "RemoveContainer" containerID="e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13"
Sep 18 18:53:21.590723 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.590683    2581 generic.go:356] "Generic (PLEG): container finished" podID="5ca964b6-de47-4e24-8f39-e316edf35898" containerID="189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc" exitCode=0
Sep 18 18:53:21.590799 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.590786    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:21.592616 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.592592    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl" event={"ID":"081fc10e-a177-4bbc-977e-41978480fb31","Type":"ContainerStarted","Data":"cf9333b4289852270f0475ddcd02a7f451c15a314a36037da296cd016e66c6f4"}
Sep 18 18:53:21.592770 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.592635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl" event={"ID":"081fc10e-a177-4bbc-977e-41978480fb31","Type":"ContainerStarted","Data":"0f352ec5386736f571854c325204e8e2c984c805950d5dba5be76d4a0fdb1a0d"}
Sep 18 18:53:21.592770 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.592750    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl"
Sep 18 18:53:21.595002 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:21.594974    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:21.600429 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:21.600394    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-smr4p\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="5ca964b6-de47-4e24-8f39-e316edf35898" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:21.601040 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.601018    2581 scope.go:117] "RemoveContainer" containerID="e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13"
Sep 18 18:53:21.601421 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:21.601401    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13\": container with ID starting with e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13 not found: ID does not exist" containerID="e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13"
Sep 18 18:53:21.601509 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.601433    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13"} err="failed to get container status \"e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13\": rpc error: code = NotFound desc = could not find container \"e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13\": container with ID starting with e9af7c7ea5394ced090bf48b0396f5bc0ebd1ed8eff70756f86413bf18756b13 not found: ID does not exist"
Sep 18 18:53:21.601509 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.601457    2581 scope.go:117] "RemoveContainer" containerID="189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc"
Sep 18 18:53:21.610040 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:21.609999    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:21.611270 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.611246    2581 scope.go:117] "RemoveContainer" containerID="189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc"
Sep 18 18:53:21.611762 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:21.611639    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc\": container with ID starting with 189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc not found: ID does not exist" containerID="189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc"
Sep 18 18:53:21.611762 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.611671    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc"} err="failed to get container status \"189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc\": rpc error: code = NotFound desc = could not find container \"189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc\": container with ID starting with 189ab9f718e8ef2cdb695a3af152dac81d0a448434b40ed6dd1c5de724ca19dc not found: ID does not exist"
Sep 18 18:53:21.613452 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:21.613430    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"]
Sep 18 18:53:21.616160 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:53:21.616136    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e990f7_49a2_4b1e_8889_862fe80e4106.slice/crio-73903a61e5de95106d61d2fa2ca41c5436ec6d0d8e68af3c0f7b8b31b05178ac WatchSource:0}: Error finding container 73903a61e5de95106d61d2fa2ca41c5436ec6d0d8e68af3c0f7b8b31b05178ac: Status 404 returned error can't find the container with id 73903a61e5de95106d61d2fa2ca41c5436ec6d0d8e68af3c0f7b8b31b05178ac
Sep 18 18:53:21.617268 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:21.617234    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-smr4p\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="5ca964b6-de47-4e24-8f39-e316edf35898" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:21.666888 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:21.666839    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s9xkc\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9xkc"
Sep 18 18:53:21.669152 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:21.669119    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-smr4p\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="5ca964b6-de47-4e24-8f39-e316edf35898" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-smr4p"
Sep 18 18:53:22.312852 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:22.312821    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ca964b6-de47-4e24-8f39-e316edf35898" path="/var/lib/kubelet/pods/5ca964b6-de47-4e24-8f39-e316edf35898/volumes"
Sep 18 18:53:22.313172 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:22.313160    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e52851-7f1f-4915-9240-243faf6b0ced" path="/var/lib/kubelet/pods/d6e52851-7f1f-4915-9240-243faf6b0ced/volumes"
Sep 18 18:53:22.610691 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:22.605193    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn" event={"ID":"f9e990f7-49a2-4b1e-8889-862fe80e4106","Type":"ContainerStarted","Data":"629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02"}
Sep 18 18:53:22.610691 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:22.605536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn" event={"ID":"f9e990f7-49a2-4b1e-8889-862fe80e4106","Type":"ContainerStarted","Data":"73903a61e5de95106d61d2fa2ca41c5436ec6d0d8e68af3c0f7b8b31b05178ac"}
Sep 18 18:53:22.610691 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:22.606280    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:22.650193 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:22.650135    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn" podStartSLOduration=2.650118538 podStartE2EDuration="2.650118538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:22.647691052 +0000 UTC m=+538.866698629" watchObservedRunningTime="2026-09-18 18:53:22.650118538 +0000 UTC m=+538.869126237"
Sep 18 18:53:22.650352 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:22.650231    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl" podStartSLOduration=2.65022551 podStartE2EDuration="2.65022551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:21.664395227 +0000 UTC m=+537.883402811" watchObservedRunningTime="2026-09-18 18:53:22.65022551 +0000 UTC m=+538.869233089"
Sep 18 18:53:24.150368 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.150325    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"]
Sep 18 18:53:24.153998 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.153975    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 18:53:24.193902 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.193860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"]
Sep 18 18:53:24.255437 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.255411    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7b4h\" (UniqueName: \"kubernetes.io/projected/7673227a-1079-400b-a86d-83de486404ad-kube-api-access-v7b4h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f244x\" (UID: \"7673227a-1079-400b-a86d-83de486404ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 18:53:24.255501 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.255480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7673227a-1079-400b-a86d-83de486404ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f244x\" (UID: \"7673227a-1079-400b-a86d-83de486404ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 18:53:24.356945 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.356902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7673227a-1079-400b-a86d-83de486404ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f244x\" (UID: \"7673227a-1079-400b-a86d-83de486404ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 18:53:24.357100 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.356954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7b4h\" (UniqueName: \"kubernetes.io/projected/7673227a-1079-400b-a86d-83de486404ad-kube-api-access-v7b4h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f244x\" (UID: \"7673227a-1079-400b-a86d-83de486404ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 18:53:24.357307 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.357287    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7673227a-1079-400b-a86d-83de486404ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f244x\" (UID: \"7673227a-1079-400b-a86d-83de486404ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 18:53:24.371870 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.371841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7b4h\" (UniqueName: \"kubernetes.io/projected/7673227a-1079-400b-a86d-83de486404ad-kube-api-access-v7b4h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f244x\" (UID: \"7673227a-1079-400b-a86d-83de486404ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 18:53:24.464184 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.464140    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 18:53:24.797570 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:24.797544    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"]
Sep 18 18:53:24.799480 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:53:24.799449    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7673227a_1079_400b_a86d_83de486404ad.slice/crio-182cc1344b34d1dfc8b48c3938379371661680ee414aa9f36047c7d302f08d8d WatchSource:0}: Error finding container 182cc1344b34d1dfc8b48c3938379371661680ee414aa9f36047c7d302f08d8d: Status 404 returned error can't find the container with id 182cc1344b34d1dfc8b48c3938379371661680ee414aa9f36047c7d302f08d8d
Sep 18 18:53:25.623598 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:25.623562    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x" event={"ID":"7673227a-1079-400b-a86d-83de486404ad","Type":"ContainerStarted","Data":"fc2c1f255eae53ca09f821e85fb4cca907152e1ee1abc6f8e16ecc32fcef160b"}
Sep 18 18:53:25.624055 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:25.623613    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x" event={"ID":"7673227a-1079-400b-a86d-83de486404ad","Type":"ContainerStarted","Data":"182cc1344b34d1dfc8b48c3938379371661680ee414aa9f36047c7d302f08d8d"}
Sep 18 18:53:25.624055 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:25.623691    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 18:53:25.686722 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:25.686628    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x" podStartSLOduration=1.686606142 podStartE2EDuration="1.686606142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:25.682816168 +0000 UTC m=+541.901823747" watchObservedRunningTime="2026-09-18 18:53:25.686606142 +0000 UTC m=+541.905613721"
Sep 18 18:53:32.607954 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:32.607919    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ddjfl"
Sep 18 18:53:34.619084 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:34.619049    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:36.631459 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:36.631423    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 18:53:36.738783 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:36.738749    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"]
Sep 18 18:53:36.738985 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:36.738943    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn" podUID="f9e990f7-49a2-4b1e-8889-862fe80e4106" containerName="manager" containerID="cri-o://629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02" gracePeriod=10
Sep 18 18:53:36.996952 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:36.996914    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:37.057065 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.057031    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e990f7-49a2-4b1e-8889-862fe80e4106-extensions-socket-volume\") pod \"f9e990f7-49a2-4b1e-8889-862fe80e4106\" (UID: \"f9e990f7-49a2-4b1e-8889-862fe80e4106\") "
Sep 18 18:53:37.057243 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.057104    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpsn6\" (UniqueName: \"kubernetes.io/projected/f9e990f7-49a2-4b1e-8889-862fe80e4106-kube-api-access-fpsn6\") pod \"f9e990f7-49a2-4b1e-8889-862fe80e4106\" (UID: \"f9e990f7-49a2-4b1e-8889-862fe80e4106\") "
Sep 18 18:53:37.057543 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.057514    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e990f7-49a2-4b1e-8889-862fe80e4106-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f9e990f7-49a2-4b1e-8889-862fe80e4106" (UID: "f9e990f7-49a2-4b1e-8889-862fe80e4106"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:53:37.059520 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.059488    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e990f7-49a2-4b1e-8889-862fe80e4106-kube-api-access-fpsn6" (OuterVolumeSpecName: "kube-api-access-fpsn6") pod "f9e990f7-49a2-4b1e-8889-862fe80e4106" (UID: "f9e990f7-49a2-4b1e-8889-862fe80e4106"). InnerVolumeSpecName "kube-api-access-fpsn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:53:37.157836 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.157797    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpsn6\" (UniqueName: \"kubernetes.io/projected/f9e990f7-49a2-4b1e-8889-862fe80e4106-kube-api-access-fpsn6\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:53:37.157836 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.157828    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e990f7-49a2-4b1e-8889-862fe80e4106-extensions-socket-volume\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:53:37.689294 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.689252    2581 generic.go:356] "Generic (PLEG): container finished" podID="f9e990f7-49a2-4b1e-8889-862fe80e4106" containerID="629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02" exitCode=0
Sep 18 18:53:37.689294 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.689297    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"
Sep 18 18:53:37.689809 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.689341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn" event={"ID":"f9e990f7-49a2-4b1e-8889-862fe80e4106","Type":"ContainerDied","Data":"629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02"}
Sep 18 18:53:37.689809 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.689399    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn" event={"ID":"f9e990f7-49a2-4b1e-8889-862fe80e4106","Type":"ContainerDied","Data":"73903a61e5de95106d61d2fa2ca41c5436ec6d0d8e68af3c0f7b8b31b05178ac"}
Sep 18 18:53:37.689809 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.689423    2581 scope.go:117] "RemoveContainer" containerID="629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02"
Sep 18 18:53:37.699241 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.699219    2581 scope.go:117] "RemoveContainer" containerID="629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02"
Sep 18 18:53:37.699605 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:53:37.699583    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02\": container with ID starting with 629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02 not found: ID does not exist" containerID="629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02"
Sep 18 18:53:37.699670 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.699615    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02"} err="failed to get container status \"629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02\": rpc error: code = NotFound desc = could not find container \"629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02\": container with ID starting with 629a8e4d66366e3a35fee10a34bd5eab2509303c702848d8b218e93be5a26a02 not found: ID does not exist"
Sep 18 18:53:37.720964 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.720917    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"]
Sep 18 18:53:37.725101 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:37.725067    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-f5lrn"]
Sep 18 18:53:38.312816 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:38.312773    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9e990f7-49a2-4b1e-8889-862fe80e4106" path="/var/lib/kubelet/pods/f9e990f7-49a2-4b1e-8889-862fe80e4106/volumes"
Sep 18 18:53:53.374809 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.374767    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"]
Sep 18 18:53:53.375219 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.375098    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e990f7-49a2-4b1e-8889-862fe80e4106" containerName="manager"
Sep 18 18:53:53.375219 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.375109    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e990f7-49a2-4b1e-8889-862fe80e4106" containerName="manager"
Sep 18 18:53:53.375219 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.375158    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9e990f7-49a2-4b1e-8889-862fe80e4106" containerName="manager"
Sep 18 18:53:53.378260 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.378236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.381204 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.381179    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-45lmz\""
Sep 18 18:53:53.381204 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.381201    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:53:53.394689 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.394657    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"]
Sep 18 18:53:53.485520 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.485479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sqb7\" (UniqueName: \"kubernetes.io/projected/a2982db7-81a2-4b40-88b5-2abdac9e0152-kube-api-access-8sqb7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.485520 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.485518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.485790 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.485541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.485790 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.485589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.485790 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.485613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2982db7-81a2-4b40-88b5-2abdac9e0152-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.485790 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.485658    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.485790 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.485683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.485790 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.485724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.485790 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.485741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586312 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586269    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586312 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586540 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586540 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586464    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586621 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sqb7\" (UniqueName: \"kubernetes.io/projected/a2982db7-81a2-4b40-88b5-2abdac9e0152-kube-api-access-8sqb7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586621 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586749 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586749 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586749 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2982db7-81a2-4b40-88b5-2abdac9e0152-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586902 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.586902 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.586834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.587120 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.587096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.587214 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.587160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.587429 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.587406    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2982db7-81a2-4b40-88b5-2abdac9e0152-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.589001 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.588980    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.589214 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.589193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.595589 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.595557    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sqb7\" (UniqueName: \"kubernetes.io/projected/a2982db7-81a2-4b40-88b5-2abdac9e0152-kube-api-access-8sqb7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.595924 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.595895    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2982db7-81a2-4b40-88b5-2abdac9e0152-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jlxh4\" (UID: \"a2982db7-81a2-4b40-88b5-2abdac9e0152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.689274 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.689223    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:53.968249 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:53.968222    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"]
Sep 18 18:53:53.970208 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:53:53.970178    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2982db7_81a2_4b40_88b5_2abdac9e0152.slice/crio-e55386d88b28bee5153a5c3147b0b99611981f8710560e93197c1c0b92b6faeb WatchSource:0}: Error finding container e55386d88b28bee5153a5c3147b0b99611981f8710560e93197c1c0b92b6faeb: Status 404 returned error can't find the container with id e55386d88b28bee5153a5c3147b0b99611981f8710560e93197c1c0b92b6faeb
Sep 18 18:53:54.780736 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:54.780679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4" event={"ID":"a2982db7-81a2-4b40-88b5-2abdac9e0152","Type":"ContainerStarted","Data":"e55386d88b28bee5153a5c3147b0b99611981f8710560e93197c1c0b92b6faeb"}
Sep 18 18:53:56.733671 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:56.733631    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:53:56.733997 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:56.733734    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:53:56.733997 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:56.733762    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:53:57.799986 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:57.799948    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4" event={"ID":"a2982db7-81a2-4b40-88b5-2abdac9e0152","Type":"ContainerStarted","Data":"7ed8bdff10a839d18437c82db49d2ba1883ada980d513abb7706590e5cf0ed00"}
Sep 18 18:53:57.830595 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:57.830539    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4" podStartSLOduration=2.069464568 podStartE2EDuration="4.830522667s" totalImagesPullingTime="2.761058099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:53:57.828835789 +0000 UTC m=+574.047843366" watchObservedRunningTime="2026-09-18 18:53:57.830522667 +0000 UTC m=+574.049530245"
Sep 18 18:53:58.689626 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:58.689581    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:58.694651 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:58.694621    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:58.805118 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:58.805087    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:58.806185 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:58.806164    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:53:59.012388 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.012303    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r66nn"]
Sep 18 18:53:59.015824 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.015802    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:53:59.018784 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.018752    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 18 18:53:59.018945 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.018841    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ldxjs\""
Sep 18 18:53:59.029637 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.029600    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r66nn"]
Sep 18 18:53:59.096792 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.096757    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r66nn"]
Sep 18 18:53:59.136078 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.136044    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a28413e3-cf08-459f-b4e0-05ea4628cd77-config-file\") pod \"limitador-limitador-8588749977-r66nn\" (UID: \"a28413e3-cf08-459f-b4e0-05ea4628cd77\") " pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:53:59.136276 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.136162    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkktt\" (UniqueName: \"kubernetes.io/projected/a28413e3-cf08-459f-b4e0-05ea4628cd77-kube-api-access-rkktt\") pod \"limitador-limitador-8588749977-r66nn\" (UID: \"a28413e3-cf08-459f-b4e0-05ea4628cd77\") " pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:53:59.236764 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.236729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a28413e3-cf08-459f-b4e0-05ea4628cd77-config-file\") pod \"limitador-limitador-8588749977-r66nn\" (UID: \"a28413e3-cf08-459f-b4e0-05ea4628cd77\") " pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:53:59.236961 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.236840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkktt\" (UniqueName: \"kubernetes.io/projected/a28413e3-cf08-459f-b4e0-05ea4628cd77-kube-api-access-rkktt\") pod \"limitador-limitador-8588749977-r66nn\" (UID: \"a28413e3-cf08-459f-b4e0-05ea4628cd77\") " pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:53:59.237434 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.237417    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a28413e3-cf08-459f-b4e0-05ea4628cd77-config-file\") pod \"limitador-limitador-8588749977-r66nn\" (UID: \"a28413e3-cf08-459f-b4e0-05ea4628cd77\") " pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:53:59.247154 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.247118    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkktt\" (UniqueName: \"kubernetes.io/projected/a28413e3-cf08-459f-b4e0-05ea4628cd77-kube-api-access-rkktt\") pod \"limitador-limitador-8588749977-r66nn\" (UID: \"a28413e3-cf08-459f-b4e0-05ea4628cd77\") " pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:53:59.326660 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.326566    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:53:59.607645 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.607559    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r66nn"]
Sep 18 18:53:59.613418 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:53:59.613386    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28413e3_cf08_459f_b4e0_05ea4628cd77.slice/crio-773b69f6d3cead95f33994c31c79cf8bf9add441efff8e7a60fca585483f97ef WatchSource:0}: Error finding container 773b69f6d3cead95f33994c31c79cf8bf9add441efff8e7a60fca585483f97ef: Status 404 returned error can't find the container with id 773b69f6d3cead95f33994c31c79cf8bf9add441efff8e7a60fca585483f97ef
Sep 18 18:53:59.811406 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.811368    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r66nn" event={"ID":"a28413e3-cf08-459f-b4e0-05ea4628cd77","Type":"ContainerStarted","Data":"773b69f6d3cead95f33994c31c79cf8bf9add441efff8e7a60fca585483f97ef"}
Sep 18 18:53:59.962240 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.962198    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h9jn9"]
Sep 18 18:53:59.967105 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.967077    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9"
Sep 18 18:53:59.970088 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.970058    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wtn2g\""
Sep 18 18:53:59.978148 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:53:59.978117    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h9jn9"]
Sep 18 18:54:00.044200 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:00.044160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95z7n\" (UniqueName: \"kubernetes.io/projected/7b203723-b723-4b78-be2f-598ee2447bc4-kube-api-access-95z7n\") pod \"authorino-7bcfd8f9cd-h9jn9\" (UID: \"7b203723-b723-4b78-be2f-598ee2447bc4\") " pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9"
Sep 18 18:54:00.144719 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:00.144680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95z7n\" (UniqueName: \"kubernetes.io/projected/7b203723-b723-4b78-be2f-598ee2447bc4-kube-api-access-95z7n\") pod \"authorino-7bcfd8f9cd-h9jn9\" (UID: \"7b203723-b723-4b78-be2f-598ee2447bc4\") " pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9"
Sep 18 18:54:00.155375 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:00.155342    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95z7n\" (UniqueName: \"kubernetes.io/projected/7b203723-b723-4b78-be2f-598ee2447bc4-kube-api-access-95z7n\") pod \"authorino-7bcfd8f9cd-h9jn9\" (UID: \"7b203723-b723-4b78-be2f-598ee2447bc4\") " pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9"
Sep 18 18:54:00.277150 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:00.277050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9"
Sep 18 18:54:00.557117 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:00.557089    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h9jn9"]
Sep 18 18:54:00.559180 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:54:00.559151    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b203723_b723_4b78_be2f_598ee2447bc4.slice/crio-f7911d6b73896039e07002cc3013ae2b578f8ddd8d0c537d091fa984a6dd7646 WatchSource:0}: Error finding container f7911d6b73896039e07002cc3013ae2b578f8ddd8d0c537d091fa984a6dd7646: Status 404 returned error can't find the container with id f7911d6b73896039e07002cc3013ae2b578f8ddd8d0c537d091fa984a6dd7646
Sep 18 18:54:00.818359 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:00.818263    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9" event={"ID":"7b203723-b723-4b78-be2f-598ee2447bc4","Type":"ContainerStarted","Data":"f7911d6b73896039e07002cc3013ae2b578f8ddd8d0c537d091fa984a6dd7646"}
Sep 18 18:54:01.825863 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:01.825806    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r66nn" event={"ID":"a28413e3-cf08-459f-b4e0-05ea4628cd77","Type":"ContainerStarted","Data":"50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca"}
Sep 18 18:54:01.826274 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:01.825925    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:54:01.845622 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:01.845565    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-r66nn" podStartSLOduration=2.469155532 podStartE2EDuration="3.845547757s" totalImagesPullingTime="1.376392225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:01.844778886 +0000 UTC m=+578.063786463" watchObservedRunningTime="2026-09-18 18:54:01.845547757 +0000 UTC m=+578.064555335"
Sep 18 18:54:03.811207 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:03.811171    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h9jn9"]
Sep 18 18:54:03.839922 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:03.839884    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9" event={"ID":"7b203723-b723-4b78-be2f-598ee2447bc4","Type":"ContainerStarted","Data":"57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6"}
Sep 18 18:54:03.855739 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:03.855656    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9" podStartSLOduration=2.44714071 podStartE2EDuration="4.855635192s" totalImagesPullingTime="2.408494482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:03.854818222 +0000 UTC m=+580.073825799" watchObservedRunningTime="2026-09-18 18:54:03.855635192 +0000 UTC m=+580.074642771"
Sep 18 18:54:04.845015 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:04.844973    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9" podUID="7b203723-b723-4b78-be2f-598ee2447bc4" containerName="authorino" containerID="cri-o://57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6" gracePeriod=30
Sep 18 18:54:05.100880 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.100807    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9"
Sep 18 18:54:05.189920 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.189874    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95z7n\" (UniqueName: \"kubernetes.io/projected/7b203723-b723-4b78-be2f-598ee2447bc4-kube-api-access-95z7n\") pod \"7b203723-b723-4b78-be2f-598ee2447bc4\" (UID: \"7b203723-b723-4b78-be2f-598ee2447bc4\") "
Sep 18 18:54:05.192222 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.192181    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b203723-b723-4b78-be2f-598ee2447bc4-kube-api-access-95z7n" (OuterVolumeSpecName: "kube-api-access-95z7n") pod "7b203723-b723-4b78-be2f-598ee2447bc4" (UID: "7b203723-b723-4b78-be2f-598ee2447bc4"). InnerVolumeSpecName "kube-api-access-95z7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:54:05.291495 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.291451    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95z7n\" (UniqueName: \"kubernetes.io/projected/7b203723-b723-4b78-be2f-598ee2447bc4-kube-api-access-95z7n\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:54:05.851297 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.851253    2581 generic.go:356] "Generic (PLEG): container finished" podID="7b203723-b723-4b78-be2f-598ee2447bc4" containerID="57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6" exitCode=0
Sep 18 18:54:05.851783 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.851328    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9"
Sep 18 18:54:05.851783 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.851335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9" event={"ID":"7b203723-b723-4b78-be2f-598ee2447bc4","Type":"ContainerDied","Data":"57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6"}
Sep 18 18:54:05.851783 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.851402    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-h9jn9" event={"ID":"7b203723-b723-4b78-be2f-598ee2447bc4","Type":"ContainerDied","Data":"f7911d6b73896039e07002cc3013ae2b578f8ddd8d0c537d091fa984a6dd7646"}
Sep 18 18:54:05.851783 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.851419    2581 scope.go:117] "RemoveContainer" containerID="57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6"
Sep 18 18:54:05.861118 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.861095    2581 scope.go:117] "RemoveContainer" containerID="57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6"
Sep 18 18:54:05.861479 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:54:05.861460    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6\": container with ID starting with 57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6 not found: ID does not exist" containerID="57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6"
Sep 18 18:54:05.861552 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.861494    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6"} err="failed to get container status \"57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6\": rpc error: code = NotFound desc = could not find container \"57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6\": container with ID starting with 57246bc01f46840a26c59d1481ef0e6d391ebcf0768a29aaf1bf0d962ff2c0c6 not found: ID does not exist"
Sep 18 18:54:05.874205 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.874160    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h9jn9"]
Sep 18 18:54:05.879855 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:05.879809    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h9jn9"]
Sep 18 18:54:06.312356 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:06.312322    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b203723-b723-4b78-be2f-598ee2447bc4" path="/var/lib/kubelet/pods/7b203723-b723-4b78-be2f-598ee2447bc4/volumes"
Sep 18 18:54:12.833415 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:12.833382    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:54:14.800894 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:14.800851    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r66nn"]
Sep 18 18:54:14.801343 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:14.801066    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-r66nn" podUID="a28413e3-cf08-459f-b4e0-05ea4628cd77" containerName="limitador" containerID="cri-o://50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca" gracePeriod=30
Sep 18 18:54:15.358810 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.358778    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:54:15.480845 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.480806    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkktt\" (UniqueName: \"kubernetes.io/projected/a28413e3-cf08-459f-b4e0-05ea4628cd77-kube-api-access-rkktt\") pod \"a28413e3-cf08-459f-b4e0-05ea4628cd77\" (UID: \"a28413e3-cf08-459f-b4e0-05ea4628cd77\") "
Sep 18 18:54:15.481028 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.480868    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a28413e3-cf08-459f-b4e0-05ea4628cd77-config-file\") pod \"a28413e3-cf08-459f-b4e0-05ea4628cd77\" (UID: \"a28413e3-cf08-459f-b4e0-05ea4628cd77\") "
Sep 18 18:54:15.481249 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.481215    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a28413e3-cf08-459f-b4e0-05ea4628cd77-config-file" (OuterVolumeSpecName: "config-file") pod "a28413e3-cf08-459f-b4e0-05ea4628cd77" (UID: "a28413e3-cf08-459f-b4e0-05ea4628cd77"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:54:15.483194 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.483156    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28413e3-cf08-459f-b4e0-05ea4628cd77-kube-api-access-rkktt" (OuterVolumeSpecName: "kube-api-access-rkktt") pod "a28413e3-cf08-459f-b4e0-05ea4628cd77" (UID: "a28413e3-cf08-459f-b4e0-05ea4628cd77"). InnerVolumeSpecName "kube-api-access-rkktt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:54:15.582143 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.582103    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkktt\" (UniqueName: \"kubernetes.io/projected/a28413e3-cf08-459f-b4e0-05ea4628cd77-kube-api-access-rkktt\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:54:15.582143 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.582134    2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a28413e3-cf08-459f-b4e0-05ea4628cd77-config-file\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:54:15.910718 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.910614    2581 generic.go:356] "Generic (PLEG): container finished" podID="a28413e3-cf08-459f-b4e0-05ea4628cd77" containerID="50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca" exitCode=0
Sep 18 18:54:15.910718 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.910691    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r66nn"
Sep 18 18:54:15.911155 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.910717    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r66nn" event={"ID":"a28413e3-cf08-459f-b4e0-05ea4628cd77","Type":"ContainerDied","Data":"50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca"}
Sep 18 18:54:15.911155 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.910804    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r66nn" event={"ID":"a28413e3-cf08-459f-b4e0-05ea4628cd77","Type":"ContainerDied","Data":"773b69f6d3cead95f33994c31c79cf8bf9add441efff8e7a60fca585483f97ef"}
Sep 18 18:54:15.911155 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.910828    2581 scope.go:117] "RemoveContainer" containerID="50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca"
Sep 18 18:54:15.922111 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.922089    2581 scope.go:117] "RemoveContainer" containerID="50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca"
Sep 18 18:54:15.922466 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:54:15.922443    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca\": container with ID starting with 50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca not found: ID does not exist" containerID="50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca"
Sep 18 18:54:15.922529 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.922474    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca"} err="failed to get container status \"50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca\": rpc error: code = NotFound desc = could not find container \"50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca\": container with ID starting with 50f2935cf7af6a4f53ebcb6730035c0148b3bc048ef2777c72baa34953b2a8ca not found: ID does not exist"
Sep 18 18:54:15.934649 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.934613    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r66nn"]
Sep 18 18:54:15.943472 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:15.943439    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r66nn"]
Sep 18 18:54:16.312968 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:16.312937    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a28413e3-cf08-459f-b4e0-05ea4628cd77" path="/var/lib/kubelet/pods/a28413e3-cf08-459f-b4e0-05ea4628cd77/volumes"
Sep 18 18:54:20.656499 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.656463    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"]
Sep 18 18:54:20.656924 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.656911    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a28413e3-cf08-459f-b4e0-05ea4628cd77" containerName="limitador"
Sep 18 18:54:20.656970 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.656930    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28413e3-cf08-459f-b4e0-05ea4628cd77" containerName="limitador"
Sep 18 18:54:20.656970 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.656955    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b203723-b723-4b78-be2f-598ee2447bc4" containerName="authorino"
Sep 18 18:54:20.656970 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.656963    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b203723-b723-4b78-be2f-598ee2447bc4" containerName="authorino"
Sep 18 18:54:20.657060 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.657019    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a28413e3-cf08-459f-b4e0-05ea4628cd77" containerName="limitador"
Sep 18 18:54:20.657060 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.657028    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b203723-b723-4b78-be2f-598ee2447bc4" containerName="authorino"
Sep 18 18:54:20.660046 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.660026    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"
Sep 18 18:54:20.662751 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.662717    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 18:54:20.662751 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.662745    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-vq48g\""
Sep 18 18:54:20.662973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.662745    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 18 18:54:20.663885 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.663864    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:54:20.675674 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.675641    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"]
Sep 18 18:54:20.829415 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.829373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x77r6\" (UniqueName: \"kubernetes.io/projected/e8c94a57-1680-4607-a4e5-5f7efc70e391-kube-api-access-x77r6\") pod \"postgres-69fb4594cf-7wlw4\" (UID: \"e8c94a57-1680-4607-a4e5-5f7efc70e391\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"
Sep 18 18:54:20.829415 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.829428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e8c94a57-1680-4607-a4e5-5f7efc70e391-data\") pod \"postgres-69fb4594cf-7wlw4\" (UID: \"e8c94a57-1680-4607-a4e5-5f7efc70e391\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"
Sep 18 18:54:20.930765 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.930693    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x77r6\" (UniqueName: \"kubernetes.io/projected/e8c94a57-1680-4607-a4e5-5f7efc70e391-kube-api-access-x77r6\") pod \"postgres-69fb4594cf-7wlw4\" (UID: \"e8c94a57-1680-4607-a4e5-5f7efc70e391\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"
Sep 18 18:54:20.930938 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.930787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e8c94a57-1680-4607-a4e5-5f7efc70e391-data\") pod \"postgres-69fb4594cf-7wlw4\" (UID: \"e8c94a57-1680-4607-a4e5-5f7efc70e391\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"
Sep 18 18:54:20.931129 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.931104    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e8c94a57-1680-4607-a4e5-5f7efc70e391-data\") pod \"postgres-69fb4594cf-7wlw4\" (UID: \"e8c94a57-1680-4607-a4e5-5f7efc70e391\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"
Sep 18 18:54:20.939217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.939174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x77r6\" (UniqueName: \"kubernetes.io/projected/e8c94a57-1680-4607-a4e5-5f7efc70e391-kube-api-access-x77r6\") pod \"postgres-69fb4594cf-7wlw4\" (UID: \"e8c94a57-1680-4607-a4e5-5f7efc70e391\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"
Sep 18 18:54:20.971174 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:20.971142    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"
Sep 18 18:54:21.252392 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:21.252203    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"]
Sep 18 18:54:21.254549 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:54:21.254511    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c94a57_1680_4607_a4e5_5f7efc70e391.slice/crio-ce23237010fdcae058bf6410a6b10c173615f8b043d3ee02a3f91bc8b7c88c1d WatchSource:0}: Error finding container ce23237010fdcae058bf6410a6b10c173615f8b043d3ee02a3f91bc8b7c88c1d: Status 404 returned error can't find the container with id ce23237010fdcae058bf6410a6b10c173615f8b043d3ee02a3f91bc8b7c88c1d
Sep 18 18:54:21.942161 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:21.942116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4" event={"ID":"e8c94a57-1680-4607-a4e5-5f7efc70e391","Type":"ContainerStarted","Data":"ce23237010fdcae058bf6410a6b10c173615f8b043d3ee02a3f91bc8b7c88c1d"}
Sep 18 18:54:26.248309 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:26.248278    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 18 18:54:26.972392 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:26.972351    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4" event={"ID":"e8c94a57-1680-4607-a4e5-5f7efc70e391","Type":"ContainerStarted","Data":"fc542c6ade254e1e5eb0158d0964c9dc2bed58b26fd162501769028079f432f1"}
Sep 18 18:54:26.972570 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:26.972439    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"
Sep 18 18:54:26.991164 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:26.991106    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4" podStartSLOduration=2.001761335 podStartE2EDuration="6.991086566s" totalImagesPullingTime="4.989325231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:26.989317082 +0000 UTC m=+603.208324660" watchObservedRunningTime="2026-09-18 18:54:26.991086566 +0000 UTC m=+603.210094144"
Sep 18 18:54:33.008958 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:33.008928    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7wlw4"
Sep 18 18:54:33.841521 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:33.841488    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-dd587cbd8-zkvjm"]
Sep 18 18:54:33.847877 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:33.847849    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dd587cbd8-zkvjm"
Sep 18 18:54:33.850528 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:33.850496    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wtn2g\""
Sep 18 18:54:33.851897 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:33.851874    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-dd587cbd8-zkvjm"]
Sep 18 18:54:33.892932 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:33.892895    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-dd587cbd8-zkvjm"]
Sep 18 18:54:33.893178 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:54:33.893158    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-dx5hw], unattached volumes=[], failed to process volumes=[kube-api-access-dx5hw]: context canceled" pod="kuadrant-system/authorino-dd587cbd8-zkvjm" podUID="361478b9-4a49-4dc3-a199-0a17c9dfee85"
Sep 18 18:54:33.945659 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:33.945618    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx5hw\" (UniqueName: \"kubernetes.io/projected/361478b9-4a49-4dc3-a199-0a17c9dfee85-kube-api-access-dx5hw\") pod \"authorino-dd587cbd8-zkvjm\" (UID: \"361478b9-4a49-4dc3-a199-0a17c9dfee85\") " pod="kuadrant-system/authorino-dd587cbd8-zkvjm"
Sep 18 18:54:34.007642 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:34.007602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dd587cbd8-zkvjm"
Sep 18 18:54:34.015961 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:34.015932    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dd587cbd8-zkvjm"
Sep 18 18:54:34.046235 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:34.046169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5hw\" (UniqueName: \"kubernetes.io/projected/361478b9-4a49-4dc3-a199-0a17c9dfee85-kube-api-access-dx5hw\") pod \"authorino-dd587cbd8-zkvjm\" (UID: \"361478b9-4a49-4dc3-a199-0a17c9dfee85\") " pod="kuadrant-system/authorino-dd587cbd8-zkvjm"
Sep 18 18:54:34.054941 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:34.054906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx5hw\" (UniqueName: \"kubernetes.io/projected/361478b9-4a49-4dc3-a199-0a17c9dfee85-kube-api-access-dx5hw\") pod \"authorino-dd587cbd8-zkvjm\" (UID: \"361478b9-4a49-4dc3-a199-0a17c9dfee85\") " pod="kuadrant-system/authorino-dd587cbd8-zkvjm"
Sep 18 18:54:34.146608 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:34.146516    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx5hw\" (UniqueName: \"kubernetes.io/projected/361478b9-4a49-4dc3-a199-0a17c9dfee85-kube-api-access-dx5hw\") pod \"361478b9-4a49-4dc3-a199-0a17c9dfee85\" (UID: \"361478b9-4a49-4dc3-a199-0a17c9dfee85\") "
Sep 18 18:54:34.149060 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:34.149020    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/361478b9-4a49-4dc3-a199-0a17c9dfee85-kube-api-access-dx5hw" (OuterVolumeSpecName: "kube-api-access-dx5hw") pod "361478b9-4a49-4dc3-a199-0a17c9dfee85" (UID: "361478b9-4a49-4dc3-a199-0a17c9dfee85"). InnerVolumeSpecName "kube-api-access-dx5hw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:54:34.247780 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:34.247745    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx5hw\" (UniqueName: \"kubernetes.io/projected/361478b9-4a49-4dc3-a199-0a17c9dfee85-kube-api-access-dx5hw\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:54:35.012771 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:35.012740    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dd587cbd8-zkvjm"
Sep 18 18:54:35.043098 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:35.043059    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-dd587cbd8-zkvjm"]
Sep 18 18:54:35.047112 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:35.047073    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-dd587cbd8-zkvjm"]
Sep 18 18:54:36.313410 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:36.313372    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="361478b9-4a49-4dc3-a199-0a17c9dfee85" path="/var/lib/kubelet/pods/361478b9-4a49-4dc3-a199-0a17c9dfee85/volumes"
Sep 18 18:54:41.018458 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.018414    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84944d87d7-bfdh2"]
Sep 18 18:54:41.022082 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.022060    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.025116 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.025085    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 18:54:41.025116 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.025102    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 18:54:41.025340 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.025258    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pcg7z\""
Sep 18 18:54:41.026191 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.026168    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 18:54:41.044956 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.044921    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84944d87d7-bfdh2"]
Sep 18 18:54:41.097384 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.097342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-webhook-cert\") pod \"maas-controller-84944d87d7-bfdh2\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") " pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.097563 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.097404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx4xc\" (UniqueName: \"kubernetes.io/projected/95a94070-857b-4c5d-9d60-7885f761c121-kube-api-access-vx4xc\") pod \"maas-controller-84944d87d7-bfdh2\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") " pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.097563 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.097431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-metrics-certs\") pod \"maas-controller-84944d87d7-bfdh2\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") " pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.198072 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.198040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-metrics-certs\") pod \"maas-controller-84944d87d7-bfdh2\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") " pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.198259 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.198098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-webhook-cert\") pod \"maas-controller-84944d87d7-bfdh2\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") " pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.198259 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.198138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4xc\" (UniqueName: \"kubernetes.io/projected/95a94070-857b-4c5d-9d60-7885f761c121-kube-api-access-vx4xc\") pod \"maas-controller-84944d87d7-bfdh2\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") " pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.200680 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.200649    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-webhook-cert\") pod \"maas-controller-84944d87d7-bfdh2\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") " pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.200825 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.200798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-metrics-certs\") pod \"maas-controller-84944d87d7-bfdh2\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") " pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.207069 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.207035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx4xc\" (UniqueName: \"kubernetes.io/projected/95a94070-857b-4c5d-9d60-7885f761c121-kube-api-access-vx4xc\") pod \"maas-controller-84944d87d7-bfdh2\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") " pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.333299 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.333210    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:41.605776 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:41.605752    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84944d87d7-bfdh2"]
Sep 18 18:54:41.608573 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:54:41.608546    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a94070_857b_4c5d_9d60_7885f761c121.slice/crio-de8fe2f5857a805e4664691ebdce5d252fbfe8532857c3a07f45cad1ba6a0cc9 WatchSource:0}: Error finding container de8fe2f5857a805e4664691ebdce5d252fbfe8532857c3a07f45cad1ba6a0cc9: Status 404 returned error can't find the container with id de8fe2f5857a805e4664691ebdce5d252fbfe8532857c3a07f45cad1ba6a0cc9
Sep 18 18:54:42.048662 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:42.048620    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84944d87d7-bfdh2" event={"ID":"95a94070-857b-4c5d-9d60-7885f761c121","Type":"ContainerStarted","Data":"de8fe2f5857a805e4664691ebdce5d252fbfe8532857c3a07f45cad1ba6a0cc9"}
Sep 18 18:54:44.068922 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:44.068882    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84944d87d7-bfdh2" event={"ID":"95a94070-857b-4c5d-9d60-7885f761c121","Type":"ContainerStarted","Data":"887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e"}
Sep 18 18:54:44.069390 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:44.069031    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:54:44.093912 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:44.093854    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84944d87d7-bfdh2" podStartSLOduration=1.782214524 podStartE2EDuration="4.093828777s" totalImagesPullingTime="2.311614253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:44.090872521 +0000 UTC m=+620.309880113" watchObservedRunningTime="2026-09-18 18:54:44.093828777 +0000 UTC m=+620.312836356"
Sep 18 18:54:55.076032 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:54:55.075998    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 18:55:17.250833 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.250797    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-868977f559-ggcg7"]
Sep 18 18:55:17.257050 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.257029    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.260930 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.260907    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-f6fxk\""
Sep 18 18:55:17.261072 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.260908    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 18:55:17.261072 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.260908    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 18:55:17.267411 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.267380    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-868977f559-ggcg7"]
Sep 18 18:55:17.387774 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.387741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-maas-api-tls\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.387960 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.387802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d7111d8-8281-4c7c-b55e-8310699a7bb0-openshift-service-ca\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.387960 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.387820    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktwpv\" (UniqueName: \"kubernetes.io/projected/8d7111d8-8281-4c7c-b55e-8310699a7bb0-kube-api-access-ktwpv\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.387960 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.387912    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-metrics-certs\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.489165 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.489124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d7111d8-8281-4c7c-b55e-8310699a7bb0-openshift-service-ca\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.489367 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.489170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktwpv\" (UniqueName: \"kubernetes.io/projected/8d7111d8-8281-4c7c-b55e-8310699a7bb0-kube-api-access-ktwpv\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.489367 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.489208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-metrics-certs\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.489367 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.489246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-maas-api-tls\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.489522 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:55:17.489374    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 18 18:55:17.489522 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:55:17.489458    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-metrics-certs podName:8d7111d8-8281-4c7c-b55e-8310699a7bb0 nodeName:}" failed. No retries permitted until 2026-09-18 18:55:17.98943384 +0000 UTC m=+654.208441412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-metrics-certs") pod "maas-api-868977f559-ggcg7" (UID: "8d7111d8-8281-4c7c-b55e-8310699a7bb0") : secret "maas-api-metrics-tls" not found
Sep 18 18:55:17.489986 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.489961    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d7111d8-8281-4c7c-b55e-8310699a7bb0-openshift-service-ca\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.491920 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.491894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-maas-api-tls\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.504362 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.504272    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktwpv\" (UniqueName: \"kubernetes.io/projected/8d7111d8-8281-4c7c-b55e-8310699a7bb0-kube-api-access-ktwpv\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.820118 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.820031    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"]
Sep 18 18:55:17.823440 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.823414    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"
Sep 18 18:55:17.826148 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.826121    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-r8zzg\""
Sep 18 18:55:17.826460 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.826223    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 18:55:17.834515 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.834485    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"]
Sep 18 18:55:17.892680 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.892639    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz22p\" (UniqueName: \"kubernetes.io/projected/c9c0d59a-eab3-4afb-ab9a-8ee23b012266-kube-api-access-cz22p\") pod \"payload-processing-5b54b7fbb7-wwd2p\" (UID: \"c9c0d59a-eab3-4afb-ab9a-8ee23b012266\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"
Sep 18 18:55:17.892879 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.892725    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9c0d59a-eab3-4afb-ab9a-8ee23b012266-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-wwd2p\" (UID: \"c9c0d59a-eab3-4afb-ab9a-8ee23b012266\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"
Sep 18 18:55:17.993830 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.993792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-metrics-certs\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:17.994011 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.993863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz22p\" (UniqueName: \"kubernetes.io/projected/c9c0d59a-eab3-4afb-ab9a-8ee23b012266-kube-api-access-cz22p\") pod \"payload-processing-5b54b7fbb7-wwd2p\" (UID: \"c9c0d59a-eab3-4afb-ab9a-8ee23b012266\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"
Sep 18 18:55:17.994011 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.993916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9c0d59a-eab3-4afb-ab9a-8ee23b012266-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-wwd2p\" (UID: \"c9c0d59a-eab3-4afb-ab9a-8ee23b012266\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"
Sep 18 18:55:17.994537 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.994518    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9c0d59a-eab3-4afb-ab9a-8ee23b012266-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-wwd2p\" (UID: \"c9c0d59a-eab3-4afb-ab9a-8ee23b012266\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"
Sep 18 18:55:17.996378 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:17.996357    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-metrics-certs\") pod \"maas-api-868977f559-ggcg7\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") " pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:18.002231 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:18.002193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz22p\" (UniqueName: \"kubernetes.io/projected/c9c0d59a-eab3-4afb-ab9a-8ee23b012266-kube-api-access-cz22p\") pod \"payload-processing-5b54b7fbb7-wwd2p\" (UID: \"c9c0d59a-eab3-4afb-ab9a-8ee23b012266\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"
Sep 18 18:55:18.136067 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:18.135968    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"
Sep 18 18:55:18.168810 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:18.168779    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:18.412083 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:18.412039    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"]
Sep 18 18:55:18.414396 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:55:18.414369    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c0d59a_eab3_4afb_ab9a_8ee23b012266.slice/crio-67c4cd46d70b04db0328bca00762c4faec46428e717fb3bcc416f23833bd8ca3 WatchSource:0}: Error finding container 67c4cd46d70b04db0328bca00762c4faec46428e717fb3bcc416f23833bd8ca3: Status 404 returned error can't find the container with id 67c4cd46d70b04db0328bca00762c4faec46428e717fb3bcc416f23833bd8ca3
Sep 18 18:55:18.416241 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:18.416225    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:55:18.467341 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:18.467159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-868977f559-ggcg7"]
Sep 18 18:55:18.470079 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:55:18.470050    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d7111d8_8281_4c7c_b55e_8310699a7bb0.slice/crio-0b4ee04920aae328bc2d344bc7cdcff66b8ae4d8ca57735f396bd08f25855ce0 WatchSource:0}: Error finding container 0b4ee04920aae328bc2d344bc7cdcff66b8ae4d8ca57735f396bd08f25855ce0: Status 404 returned error can't find the container with id 0b4ee04920aae328bc2d344bc7cdcff66b8ae4d8ca57735f396bd08f25855ce0
Sep 18 18:55:19.260792 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:19.260745    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7" event={"ID":"8d7111d8-8281-4c7c-b55e-8310699a7bb0","Type":"ContainerStarted","Data":"0b4ee04920aae328bc2d344bc7cdcff66b8ae4d8ca57735f396bd08f25855ce0"}
Sep 18 18:55:19.262938 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:19.262907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p" event={"ID":"c9c0d59a-eab3-4afb-ab9a-8ee23b012266","Type":"ContainerStarted","Data":"67c4cd46d70b04db0328bca00762c4faec46428e717fb3bcc416f23833bd8ca3"}
Sep 18 18:55:21.276795 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:21.276744    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p" event={"ID":"c9c0d59a-eab3-4afb-ab9a-8ee23b012266","Type":"ContainerStarted","Data":"b758864e4a2ea08369fcc83c814f02106070eb34c6a03ff7d1b4b4fc1b418fc2"}
Sep 18 18:55:21.277244 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:21.276920    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"
Sep 18 18:55:21.278463 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:21.278433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7" event={"ID":"8d7111d8-8281-4c7c-b55e-8310699a7bb0","Type":"ContainerStarted","Data":"f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f"}
Sep 18 18:55:21.278580 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:21.278493    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:21.292516 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:21.292447    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p" podStartSLOduration=2.185650489 podStartE2EDuration="4.292425561s" totalImagesPullingTime="2.106775072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:55:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:55:21.292055856 +0000 UTC m=+657.511063434" watchObservedRunningTime="2026-09-18 18:55:21.292425561 +0000 UTC m=+657.511433133"
Sep 18 18:55:21.308534 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:21.308448    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7" podStartSLOduration=2.240479254 podStartE2EDuration="4.308429948s" totalImagesPullingTime="2.067950694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:55:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:55:21.30750944 +0000 UTC m=+657.526517029" watchObservedRunningTime="2026-09-18 18:55:21.308429948 +0000 UTC m=+657.527437526"
Sep 18 18:55:27.290574 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:27.290501    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 18:55:27.313096 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:27.313004    2581 generic.go:356] "Generic (PLEG): container finished" podID="a2982db7-81a2-4b40-88b5-2abdac9e0152" containerID="7ed8bdff10a839d18437c82db49d2ba1883ada980d513abb7706590e5cf0ed00" exitCode=0
Sep 18 18:55:27.313096 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:27.313077    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4" event={"ID":"a2982db7-81a2-4b40-88b5-2abdac9e0152","Type":"ContainerDied","Data":"7ed8bdff10a839d18437c82db49d2ba1883ada980d513abb7706590e5cf0ed00"}
Sep 18 18:55:27.313525 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:27.313507    2581 scope.go:117] "RemoveContainer" containerID="7ed8bdff10a839d18437c82db49d2ba1883ada980d513abb7706590e5cf0ed00"
Sep 18 18:55:27.314577 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:27.314543    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:55:27.314639 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:27.314616    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:55:27.314673 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:27.314651    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:55:28.321492 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:28.321452    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4" event={"ID":"a2982db7-81a2-4b40-88b5-2abdac9e0152","Type":"ContainerStarted","Data":"020d2e775f4a8f58d2d8a1eac29619942006150387ba48b41c5ff1edca276fff"}
Sep 18 18:55:28.690417 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:28.690379    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:55:28.691594 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:28.691562    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jlxh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:55:28.691730 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:28.691647    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4" podUID="a2982db7-81a2-4b40-88b5-2abdac9e0152" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 18 18:55:28.805844 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:28.805798    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:55:29.689879 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:29.689831    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jlxh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:55:29.690278 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:29.689898    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4" podUID="a2982db7-81a2-4b40-88b5-2abdac9e0152" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 18 18:55:30.689826 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:30.689792    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jlxh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:55:30.690018 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:30.689862    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4" podUID="a2982db7-81a2-4b40-88b5-2abdac9e0152" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 18 18:55:31.690486 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:31.690442    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jlxh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:55:31.690963 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:31.690506    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4" podUID="a2982db7-81a2-4b40-88b5-2abdac9e0152" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 18 18:55:32.287168 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:32.287129    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwd2p"
Sep 18 18:55:32.690476 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:32.690442    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jlxh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:55:32.690637 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:32.690515    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4" podUID="a2982db7-81a2-4b40-88b5-2abdac9e0152" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 18 18:55:33.693816 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:33.693779    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:55:33.694786 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:33.694767    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jlxh4"
Sep 18 18:55:51.185674 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.185629    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"]
Sep 18 18:55:51.197430 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.197395    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.200787 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.200750    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"]
Sep 18 18:55:51.201492 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.201463    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 18:55:51.201492 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.201463    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lfvwt\""
Sep 18 18:55:51.201984 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.201624    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 18 18:55:51.201984 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.201939    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 18:55:51.285696 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.285659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krtth\" (UniqueName: \"kubernetes.io/projected/e810a820-b28d-48db-ab41-d8578f22fb47-kube-api-access-krtth\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.285953 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.285755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e810a820-b28d-48db-ab41-d8578f22fb47-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.285953 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.285777    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.285953 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.285793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.285953 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.285825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.285953 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.285885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.386745 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.386662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.386942 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.386765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.386942 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.386788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.386942 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.386838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krtth\" (UniqueName: \"kubernetes.io/projected/e810a820-b28d-48db-ab41-d8578f22fb47-kube-api-access-krtth\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.386942 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.386891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e810a820-b28d-48db-ab41-d8578f22fb47-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.386942 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.386918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.387204 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.387169    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.387204 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.387189    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.387295 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.387215    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.389272 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.389247    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e810a820-b28d-48db-ab41-d8578f22fb47-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.389516 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.389498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e810a820-b28d-48db-ab41-d8578f22fb47-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.396228 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.396200    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krtth\" (UniqueName: \"kubernetes.io/projected/e810a820-b28d-48db-ab41-d8578f22fb47-kube-api-access-krtth\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp\" (UID: \"e810a820-b28d-48db-ab41-d8578f22fb47\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.509931 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.509821    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:55:51.784160 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:51.784129    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"]
Sep 18 18:55:51.786167 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:55:51.786129    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode810a820_b28d_48db_ab41_d8578f22fb47.slice/crio-ce30014e3be687e8a2075a032821338c1d3b21360f4cf65f4b2d0baf52b33c8e WatchSource:0}: Error finding container ce30014e3be687e8a2075a032821338c1d3b21360f4cf65f4b2d0baf52b33c8e: Status 404 returned error can't find the container with id ce30014e3be687e8a2075a032821338c1d3b21360f4cf65f4b2d0baf52b33c8e
Sep 18 18:55:52.462667 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:52.462627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp" event={"ID":"e810a820-b28d-48db-ab41-d8578f22fb47","Type":"ContainerStarted","Data":"ce30014e3be687e8a2075a032821338c1d3b21360f4cf65f4b2d0baf52b33c8e"}
Sep 18 18:55:53.471335 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:53.471298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp" event={"ID":"e810a820-b28d-48db-ab41-d8578f22fb47","Type":"ContainerStarted","Data":"081453751f24792af69e566bf440f0d39511d0785ad6cc6e2b04d47c25b28f04"}
Sep 18 18:55:53.501808 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:55:53.501752    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp" podStartSLOduration=1.353307415 podStartE2EDuration="2.501733892s" totalImagesPullingTime="1.148426477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:55:53.500481939 +0000 UTC m=+689.719489518" watchObservedRunningTime="2026-09-18 18:55:53.501733892 +0000 UTC m=+689.720741469"
Sep 18 18:56:01.510209 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:01.510180    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:56:01.510774 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:01.510262    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:56:01.523176 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:01.523142    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:56:02.529002 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:02.528974    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp"
Sep 18 18:56:22.090031 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.089931    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"]
Sep 18 18:56:22.093489 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.093462    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.096523 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.096499    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 18 18:56:22.102327 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.102301    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"]
Sep 18 18:56:22.138687 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.138649    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.138687 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.138688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57505a2c-7af7-4616-9047-ae9ee25f6c2d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.138973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.138743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnrrk\" (UniqueName: \"kubernetes.io/projected/57505a2c-7af7-4616-9047-ae9ee25f6c2d-kube-api-access-cnrrk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.138973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.138779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.138973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.138827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.138973 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.138884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.240365 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.240329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.240559 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.240390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.240559 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.240418    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57505a2c-7af7-4616-9047-ae9ee25f6c2d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.240559 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.240453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnrrk\" (UniqueName: \"kubernetes.io/projected/57505a2c-7af7-4616-9047-ae9ee25f6c2d-kube-api-access-cnrrk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.240559 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.240482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.240559 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.240512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.240886 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.240857    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.240982 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.240894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.241115 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.241090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.243573 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.243546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57505a2c-7af7-4616-9047-ae9ee25f6c2d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.244015 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.243994    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57505a2c-7af7-4616-9047-ae9ee25f6c2d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.249229 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.249197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnrrk\" (UniqueName: \"kubernetes.io/projected/57505a2c-7af7-4616-9047-ae9ee25f6c2d-kube-api-access-cnrrk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8m5fh\" (UID: \"57505a2c-7af7-4616-9047-ae9ee25f6c2d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.405303 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.405199    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:22.682586 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:22.682557    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"]
Sep 18 18:56:22.684472 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:56:22.684443    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57505a2c_7af7_4616_9047_ae9ee25f6c2d.slice/crio-2ffac05a30cf947fcfeb84c7969ec0ee20d129d992a1a121f7f2fef9a68e54f1 WatchSource:0}: Error finding container 2ffac05a30cf947fcfeb84c7969ec0ee20d129d992a1a121f7f2fef9a68e54f1: Status 404 returned error can't find the container with id 2ffac05a30cf947fcfeb84c7969ec0ee20d129d992a1a121f7f2fef9a68e54f1
Sep 18 18:56:23.637957 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:23.637923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh" event={"ID":"57505a2c-7af7-4616-9047-ae9ee25f6c2d","Type":"ContainerStarted","Data":"623a9648b184bb9454c7209d409bfb26e182a67537630fca082c07db8444a70c"}
Sep 18 18:56:23.638436 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:23.637982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh" event={"ID":"57505a2c-7af7-4616-9047-ae9ee25f6c2d","Type":"ContainerStarted","Data":"2ffac05a30cf947fcfeb84c7969ec0ee20d129d992a1a121f7f2fef9a68e54f1"}
Sep 18 18:56:23.660791 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:23.660736    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh" podStartSLOduration=1.492689583 podStartE2EDuration="1.660688928s" totalImagesPullingTime="167.999345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:56:23.659520968 +0000 UTC m=+719.878528546" watchObservedRunningTime="2026-09-18 18:56:23.660688928 +0000 UTC m=+719.879696560"
Sep 18 18:56:32.406375 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:32.406330    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:32.407006 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:32.406416    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:32.419347 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:32.419322    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:56:32.696508 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:56:32.696470    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8m5fh"
Sep 18 18:58:47.482550 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.482503    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"]
Sep 18 18:58:47.486303 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.486279    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.500054 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.500023    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-1a444c-openshift-default-dockercfg-h9x7g\""
Sep 18 18:58:47.519629 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.519589    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"]
Sep 18 18:58:47.553670 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.553627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-envoy\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.553908 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.553689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-podinfo\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.553908 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.553747    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwxm8\" (UniqueName: \"kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-kube-api-access-hwxm8\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.553908 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.553801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-data\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.553908 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.553843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-socket\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.554332 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.553987    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-certs\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.554332 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.554050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aecaff8-c950-4891-937d-d8a5f87531c2-istiod-ca-cert\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.554332 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.554091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-credential-socket\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.554332 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.554124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-token\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655086 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-socket\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655086 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-certs\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655354 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aecaff8-c950-4891-937d-d8a5f87531c2-istiod-ca-cert\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655354 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-credential-socket\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655354 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-token\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655522 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655388    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-envoy\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655522 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-podinfo\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655522 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655474    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwxm8\" (UniqueName: \"kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-kube-api-access-hwxm8\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655683 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-data\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655683 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655535    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-certs\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.655683 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.655547    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-socket\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.656217 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.656189    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-data\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.656580 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.656354    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aecaff8-c950-4891-937d-d8a5f87531c2-istiod-ca-cert\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.656696 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.656423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-credential-socket\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.658466 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.658397    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-envoy\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.658643 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.658574    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-podinfo\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.677009 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.676976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwxm8\" (UniqueName: \"kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-kube-api-access-hwxm8\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.677476 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.677445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-token\") pod \"e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") " pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:47.797132 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:47.797024    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:48.084146 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:48.084116    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"]
Sep 18 18:58:48.085798 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:58:48.085767    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aecaff8_c950_4891_937d_d8a5f87531c2.slice/crio-753bbfbf739a9a9c913dd8c58f5453068839785dc05ed70db29d1670de917421 WatchSource:0}: Error finding container 753bbfbf739a9a9c913dd8c58f5453068839785dc05ed70db29d1670de917421: Status 404 returned error can't find the container with id 753bbfbf739a9a9c913dd8c58f5453068839785dc05ed70db29d1670de917421
Sep 18 18:58:48.087798 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:48.087766    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:58:48.087897 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:48.087848    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:58:48.087897 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:48.087892    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:58:48.414618 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:48.414580    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" event={"ID":"7aecaff8-c950-4891-937d-d8a5f87531c2","Type":"ContainerStarted","Data":"ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f"}
Sep 18 18:58:48.414847 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:48.414645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" event={"ID":"7aecaff8-c950-4891-937d-d8a5f87531c2","Type":"ContainerStarted","Data":"753bbfbf739a9a9c913dd8c58f5453068839785dc05ed70db29d1670de917421"}
Sep 18 18:58:48.504740 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:48.504637    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" podStartSLOduration=1.504611124 podStartE2EDuration="1.504611124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:48.502916142 +0000 UTC m=+864.721923720" watchObservedRunningTime="2026-09-18 18:58:48.504611124 +0000 UTC m=+864.723618705"
Sep 18 18:58:48.797688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:48.797598    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:49.802727 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:49.802678    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:50.425338 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:50.425293    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:50.426356 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:50.426327    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:58:53.057335 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.057288    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"]
Sep 18 18:58:53.061234 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.061205    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.065521 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.065488    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-15c3ed-serving-cert\""
Sep 18 18:58:53.083530 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.083484    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"]
Sep 18 18:58:53.207583 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.207541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtq89\" (UniqueName: \"kubernetes.io/projected/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-kube-api-access-xtq89\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.207583 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.207591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-openshift-service-ca\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.207860 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.207649    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-metrics-certs\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.207860 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.207753    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-maas-api-tls\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.308580 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.308484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-metrics-certs\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.308580 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.308545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-maas-api-tls\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.308821 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.308594    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtq89\" (UniqueName: \"kubernetes.io/projected/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-kube-api-access-xtq89\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.308821 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.308622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-openshift-service-ca\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.309284 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.309260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-openshift-service-ca\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.311356 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.311335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-maas-api-tls\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.311464 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.311377    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-metrics-certs\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.345281 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.345244    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtq89\" (UniqueName: \"kubernetes.io/projected/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-kube-api-access-xtq89\") pod \"maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.376298 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.376256    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:53.693013 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:53.691626    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"]
Sep 18 18:58:53.693013 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:58:53.691782    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod153e5cf1_aaaf_4f39_b2f1_bbe42379babe.slice/crio-fc05c9abc3ac995435bab5451c2443c1930eef5e54937ce7e494e0e7505836a5 WatchSource:0}: Error finding container fc05c9abc3ac995435bab5451c2443c1930eef5e54937ce7e494e0e7505836a5: Status 404 returned error can't find the container with id fc05c9abc3ac995435bab5451c2443c1930eef5e54937ce7e494e0e7505836a5
Sep 18 18:58:54.456486 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:54.456449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l" event={"ID":"153e5cf1-aaaf-4f39-b2f1-bbe42379babe","Type":"ContainerStarted","Data":"8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee"}
Sep 18 18:58:54.457017 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:54.456508    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l" event={"ID":"153e5cf1-aaaf-4f39-b2f1-bbe42379babe","Type":"ContainerStarted","Data":"fc05c9abc3ac995435bab5451c2443c1930eef5e54937ce7e494e0e7505836a5"}
Sep 18 18:58:54.457017 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:54.456568    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:58:54.530551 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:54.530485    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l" podStartSLOduration=1.079523522 podStartE2EDuration="1.530462137s" totalImagesPullingTime="450.938615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:54.527964077 +0000 UTC m=+870.746971655" watchObservedRunningTime="2026-09-18 18:58:54.530462137 +0000 UTC m=+870.749469716"
Sep 18 18:58:55.395134 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.395097    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"]
Sep 18 18:58:55.398771 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.398739    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.415793 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.415766    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-942883-serving-cert\""
Sep 18 18:58:55.449291 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.449255    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"]
Sep 18 18:58:55.529956 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.529922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-maas-api-tls\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.529956 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.529956    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-metrics-certs\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.530489 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.529997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmzp2\" (UniqueName: \"kubernetes.io/projected/57af070a-8c05-4025-a32a-c90c13c29f52-kube-api-access-pmzp2\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.530489 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.530153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57af070a-8c05-4025-a32a-c90c13c29f52-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.631285 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.631247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-maas-api-tls\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.631285 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.631284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-metrics-certs\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.631544 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.631306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmzp2\" (UniqueName: \"kubernetes.io/projected/57af070a-8c05-4025-a32a-c90c13c29f52-kube-api-access-pmzp2\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.631544 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.631343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57af070a-8c05-4025-a32a-c90c13c29f52-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.632007 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.631984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57af070a-8c05-4025-a32a-c90c13c29f52-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.634201 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.634171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-metrics-certs\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.634294 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.634270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-maas-api-tls\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.652190 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.652115    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmzp2\" (UniqueName: \"kubernetes.io/projected/57af070a-8c05-4025-a32a-c90c13c29f52-kube-api-access-pmzp2\") pod \"maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:55.710467 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:55.710424    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:56.013744 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:56.013715    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"]
Sep 18 18:58:56.016191 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:58:56.016157    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57af070a_8c05_4025_a32a_c90c13c29f52.slice/crio-a7815e760d1c057e063ad3330b023e18f773eceaae7f1676269256e754bf3a5b WatchSource:0}: Error finding container a7815e760d1c057e063ad3330b023e18f773eceaae7f1676269256e754bf3a5b: Status 404 returned error can't find the container with id a7815e760d1c057e063ad3330b023e18f773eceaae7f1676269256e754bf3a5b
Sep 18 18:58:56.477299 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:56.477184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh" event={"ID":"57af070a-8c05-4025-a32a-c90c13c29f52","Type":"ContainerStarted","Data":"a7815e760d1c057e063ad3330b023e18f773eceaae7f1676269256e754bf3a5b"}
Sep 18 18:58:57.265878 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.265832    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"]
Sep 18 18:58:57.269566 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.269537    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.281161 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.281136    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-fb8843-openshift-default-dockercfg-5dsd6\""
Sep 18 18:58:57.318042 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.317996    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"]
Sep 18 18:58:57.351094 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.351054    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-credential-socket\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.351267 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.351103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-certs\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.351267 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.351135    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-envoy\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.351267 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.351215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istiod-ca-cert\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.351267 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.351256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-data\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.351475 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.351291    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-token\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.351475 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.351328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-socket\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.351475 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.351363    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plj25\" (UniqueName: \"kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-kube-api-access-plj25\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.351475 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.351418    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-podinfo\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.452959 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.452911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-credential-socket\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453153 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-certs\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453153 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-envoy\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453153 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istiod-ca-cert\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453153 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453101    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-data\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453153 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453116    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-token\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453153 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-socket\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453452 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453184    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plj25\" (UniqueName: \"kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-kube-api-access-plj25\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453452 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453237    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-podinfo\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453452 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453407    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-certs\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453452 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-credential-socket\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453665 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-data\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.453768 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.453670    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-socket\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.454063 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.454039    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istiod-ca-cert\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.455607 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.455581    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-envoy\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.455826 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.455805    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-podinfo\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.487849 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.487811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh" event={"ID":"57af070a-8c05-4025-a32a-c90c13c29f52","Type":"ContainerStarted","Data":"1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059"}
Sep 18 18:58:57.488040 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.487931    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:58:57.489768 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.489732    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-token\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.503908 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.503874    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plj25\" (UniqueName: \"kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-kube-api-access-plj25\") pod \"e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") " pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.580812 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.580682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:57.659296 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.659238    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh" podStartSLOduration=2.267112206 podStartE2EDuration="2.65921898s" totalImagesPullingTime="392.106774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:57.584597072 +0000 UTC m=+873.803604650" watchObservedRunningTime="2026-09-18 18:58:57.65921898 +0000 UTC m=+873.878226559"
Sep 18 18:58:57.901027 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.900926    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"]
Sep 18 18:58:57.918378 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:58:57.918337    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod403a3ed9_8a91_4ca1_828b_64c0ca0741a9.slice/crio-e528ac51822b3a294935bc0a65e1d6720ebefc6d2388d19235d4f022343f0ce3 WatchSource:0}: Error finding container e528ac51822b3a294935bc0a65e1d6720ebefc6d2388d19235d4f022343f0ce3: Status 404 returned error can't find the container with id e528ac51822b3a294935bc0a65e1d6720ebefc6d2388d19235d4f022343f0ce3
Sep 18 18:58:57.921037 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.921002    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:58:57.921162 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.921077    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:58:57.921162 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:57.921111    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:58:58.495373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:58.495329    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" event={"ID":"403a3ed9-8a91-4ca1-828b-64c0ca0741a9","Type":"ContainerStarted","Data":"39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a"}
Sep 18 18:58:58.495892 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:58.495387    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" event={"ID":"403a3ed9-8a91-4ca1-828b-64c0ca0741a9","Type":"ContainerStarted","Data":"e528ac51822b3a294935bc0a65e1d6720ebefc6d2388d19235d4f022343f0ce3"}
Sep 18 18:58:58.540032 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:58.539972    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" podStartSLOduration=1.53995396 podStartE2EDuration="1.53995396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:58.539388351 +0000 UTC m=+874.758395929" watchObservedRunningTime="2026-09-18 18:58:58.53995396 +0000 UTC m=+874.758961534"
Sep 18 18:58:58.581636 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:58.581588    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:58.586972 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:58.586942    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:59.500543 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:59.500510    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:58:59.501576 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:58:59.501550    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:59:00.469556 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:00.469525    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 18:59:03.501661 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:03.501631    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 18:59:04.169170 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.169131    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"]
Sep 18 18:59:04.172948 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.172929    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.186770 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.186743    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8cbeacf3-openshift-default-dockercfg-jtdtv\""
Sep 18 18:59:04.253762 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.253723    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"]
Sep 18 18:59:04.324664 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.324631    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-certs\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.324870 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.324674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-socket\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.324870 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.324696    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmj8v\" (UniqueName: \"kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-kube-api-access-nmj8v\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.324870 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.324801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115a52cd-9845-4bf3-8577-557fbda14e55-istio-podinfo\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.325076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.324871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-istio-token\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.325076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.324908    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-credential-socket\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.325076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.324933    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-envoy\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.325076 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.325061    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-data\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.325292 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.325081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115a52cd-9845-4bf3-8577-557fbda14e55-istiod-ca-cert\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.426282 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.426182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-certs\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.426282 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.426230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-socket\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.426282 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.426248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmj8v\" (UniqueName: \"kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-kube-api-access-nmj8v\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.426282 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.426266    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115a52cd-9845-4bf3-8577-557fbda14e55-istio-podinfo\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.426620 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.426294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-istio-token\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.426620 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.426517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-credential-socket\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.426620 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.426545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-envoy\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.426801 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.426685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-certs\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.426937 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.426913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-credential-socket\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.427078 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.427056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-data\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.427255 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.427236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115a52cd-9845-4bf3-8577-557fbda14e55-istiod-ca-cert\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.427553 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.427301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-data\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.427727 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.427123    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-socket\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.427862 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.427844    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115a52cd-9845-4bf3-8577-557fbda14e55-istiod-ca-cert\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.429337 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.429305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-envoy\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.429444 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.429427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115a52cd-9845-4bf3-8577-557fbda14e55-istio-podinfo\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.460940 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.460900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmj8v\" (UniqueName: \"kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-kube-api-access-nmj8v\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.461224 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.461201    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-istio-token\") pod \"e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") " pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.483849 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.483810    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:04.816181 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.816153    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"]
Sep 18 18:59:04.818466 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:59:04.818439    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115a52cd_9845_4bf3_8577_557fbda14e55.slice/crio-62298dfe1bdee6e87fc6d6e201d6c5a36092d04a2a9263c0b648badad0813ff9 WatchSource:0}: Error finding container 62298dfe1bdee6e87fc6d6e201d6c5a36092d04a2a9263c0b648badad0813ff9: Status 404 returned error can't find the container with id 62298dfe1bdee6e87fc6d6e201d6c5a36092d04a2a9263c0b648badad0813ff9
Sep 18 18:59:04.820716 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.820673    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:04.820775 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.820758    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:04.820807 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:04.820798    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:05.538638 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.538593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" event={"ID":"115a52cd-9845-4bf3-8577-557fbda14e55","Type":"ContainerStarted","Data":"25d979c60200ed7f16e2bb5d15830b5ae452f728468543b7a27cb94bd526d236"}
Sep 18 18:59:05.538883 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.538734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" event={"ID":"115a52cd-9845-4bf3-8577-557fbda14e55","Type":"ContainerStarted","Data":"62298dfe1bdee6e87fc6d6e201d6c5a36092d04a2a9263c0b648badad0813ff9"}
Sep 18 18:59:05.598186 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.598133    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" podStartSLOduration=1.598114027 podStartE2EDuration="1.598114027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:05.594566284 +0000 UTC m=+881.813573906" watchObservedRunningTime="2026-09-18 18:59:05.598114027 +0000 UTC m=+881.817121607"
Sep 18 18:59:05.657218 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.657175    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"]
Sep 18 18:59:05.661041 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.661017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.663857 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:59:05.663821    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="9de0f90e-7420-4853-ba27-bf9b433ea545" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.664227 ip-10-0-136-52 kubenswrapper[2581]: E0918 18:59:05.664200    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-autores-fb8843-serving-cert\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-fb8843-serving-cert\"" type="*v1.Secret"
Sep 18 18:59:05.699314 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.699277    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"]
Sep 18 18:59:05.839626 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.839525    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de0f90e-7420-4853-ba27-bf9b433ea545-openshift-service-ca\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.839626 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.839571    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-maas-api-tls\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.839626 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.839589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfxq2\" (UniqueName: \"kubernetes.io/projected/9de0f90e-7420-4853-ba27-bf9b433ea545-kube-api-access-hfxq2\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.840147 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.839680    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-metrics-certs\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.941081 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.941037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de0f90e-7420-4853-ba27-bf9b433ea545-openshift-service-ca\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.941081 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.941086    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-maas-api-tls\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.941318 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.941116    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfxq2\" (UniqueName: \"kubernetes.io/projected/9de0f90e-7420-4853-ba27-bf9b433ea545-kube-api-access-hfxq2\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.941318 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.941147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-metrics-certs\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.941859 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.941834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de0f90e-7420-4853-ba27-bf9b433ea545-openshift-service-ca\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.943869 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.943848    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-metrics-certs\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:05.958969 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:05.958936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfxq2\" (UniqueName: \"kubernetes.io/projected/9de0f90e-7420-4853-ba27-bf9b433ea545-kube-api-access-hfxq2\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:06.484129 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:06.484084    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:06.489598 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:06.489569    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:06.544697 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:06.544665    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:06.545569 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:06.545548    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:06.907734 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:06.907615    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-fb8843-serving-cert\""
Sep 18 18:59:06.914535 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:06.914445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-maas-api-tls\") pod \"maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:07.173037 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:07.172990    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:07.589813 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:07.589690    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"]
Sep 18 18:59:07.596586 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:59:07.596552    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de0f90e_7420_4853_ba27_bf9b433ea545.slice/crio-0f1b97e99ffcaa6cc0cdfb7c574f8ccaf93572632e3d6a4e56fcce865f6c500c WatchSource:0}: Error finding container 0f1b97e99ffcaa6cc0cdfb7c574f8ccaf93572632e3d6a4e56fcce865f6c500c: Status 404 returned error can't find the container with id 0f1b97e99ffcaa6cc0cdfb7c574f8ccaf93572632e3d6a4e56fcce865f6c500c
Sep 18 18:59:08.560394 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:08.560345    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9" event={"ID":"9de0f90e-7420-4853-ba27-bf9b433ea545","Type":"ContainerStarted","Data":"4e395356b0d6318c3087bb5546b1cd4034c8f777b0bd0ee5db87774d31594f28"}
Sep 18 18:59:08.560829 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:08.560412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9" event={"ID":"9de0f90e-7420-4853-ba27-bf9b433ea545","Type":"ContainerStarted","Data":"0f1b97e99ffcaa6cc0cdfb7c574f8ccaf93572632e3d6a4e56fcce865f6c500c"}
Sep 18 18:59:08.560829 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:08.560438    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:08.605124 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:08.605054    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9" podStartSLOduration=3.108960723 podStartE2EDuration="3.605031232s" totalImagesPullingTime="496.070509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:08.60464136 +0000 UTC m=+884.823648964" watchObservedRunningTime="2026-09-18 18:59:08.605031232 +0000 UTC m=+884.824038810"
Sep 18 18:59:14.571499 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:14.571464    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 18:59:20.427135 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.427100    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:59:20.427595 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.427192    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 18:59:20.632354 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.632311    2581 generic.go:356] "Generic (PLEG): container finished" podID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerID="ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f" exitCode=0
Sep 18 18:59:20.632581 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.632391    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" event={"ID":"7aecaff8-c950-4891-937d-d8a5f87531c2","Type":"ContainerDied","Data":"ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f"}
Sep 18 18:59:20.632972 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.632944    2581 scope.go:117] "RemoveContainer" containerID="ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f"
Sep 18 18:59:20.634436 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.634404    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:20.634513 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.634497    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:20.634561 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.634547    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:20.958665 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.958566    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"]
Sep 18 18:59:20.962305 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.962277    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:20.978431 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:20.978403    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3a81e5-serving-cert\""
Sep 18 18:59:21.000189 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.000153    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"]
Sep 18 18:59:21.120374 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.120327    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-maas-api-tls\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.120374 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.120369    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4hdw\" (UniqueName: \"kubernetes.io/projected/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-kube-api-access-t4hdw\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.120625 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.120438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.120625 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.120487    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-metrics-certs\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.221940 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.221838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-maas-api-tls\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.221940 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.221885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4hdw\" (UniqueName: \"kubernetes.io/projected/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-kube-api-access-t4hdw\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.222164 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.222005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.222164 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.222053    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-metrics-certs\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.222741 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.222691    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.224545 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.224522    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-metrics-certs\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.224645 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.224617    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-maas-api-tls\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.246817 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.246775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4hdw\" (UniqueName: \"kubernetes.io/projected/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-kube-api-access-t4hdw\") pod \"maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.274159 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.274120    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:21.645319 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.645141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" event={"ID":"7aecaff8-c950-4891-937d-d8a5f87531c2","Type":"ContainerStarted","Data":"6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e"}
Sep 18 18:59:21.724588 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.724552    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"]
Sep 18 18:59:21.729233 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:59:21.729037    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f9884b_c729_47bd_aa5e_3e1ad77a18f2.slice/crio-54e064d0f835699c80184a66e5f27821554ed92dd9ebf15d89a49ffa22055e48 WatchSource:0}: Error finding container 54e064d0f835699c80184a66e5f27821554ed92dd9ebf15d89a49ffa22055e48: Status 404 returned error can't find the container with id 54e064d0f835699c80184a66e5f27821554ed92dd9ebf15d89a49ffa22055e48
Sep 18 18:59:21.798140 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.798105    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:59:21.798448 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.798416    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:59:21.798576 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:21.798469    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 18:59:22.658366 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:22.658329    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q" event={"ID":"11f9884b-c729-47bd-aa5e-3e1ad77a18f2","Type":"ContainerStarted","Data":"78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b"}
Sep 18 18:59:22.658975 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:22.658949    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q" event={"ID":"11f9884b-c729-47bd-aa5e-3e1ad77a18f2","Type":"ContainerStarted","Data":"54e064d0f835699c80184a66e5f27821554ed92dd9ebf15d89a49ffa22055e48"}
Sep 18 18:59:22.659138 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:22.659123    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:22.721907 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:22.721848    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q" podStartSLOduration=2.407782045 podStartE2EDuration="2.721817946s" totalImagesPullingTime="314.035901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:22.719543642 +0000 UTC m=+898.938551232" watchObservedRunningTime="2026-09-18 18:59:22.721817946 +0000 UTC m=+898.940825524"
Sep 18 18:59:22.798049 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:22.798006    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:59:22.798261 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:22.798086    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 18:59:23.798192 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:23.798151    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:59:23.798843 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:23.798810    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 18:59:24.798343 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:24.798290    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:59:24.798898 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:24.798382    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 18:59:25.798518 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:25.798472    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:59:25.799064 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:25.798542    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 18:59:26.798633 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:26.798135    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:59:26.798633 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:26.798209    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 18:59:27.802405 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:27.802369    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:59:27.802846 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:27.802672    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:59:27.803419 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:27.803396    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 18:59:29.685172 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:29.685129    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 18:59:30.501209 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:30.501167    2581 patch_prober.go:28] interesting pod/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 18:59:30.501404 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:30.501285    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 18:59:31.731345 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:31.731289    2581 generic.go:356] "Generic (PLEG): container finished" podID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerID="39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a" exitCode=0
Sep 18 18:59:31.731942 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:31.731354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" event={"ID":"403a3ed9-8a91-4ca1-828b-64c0ca0741a9","Type":"ContainerDied","Data":"39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a"}
Sep 18 18:59:31.731942 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:31.731882    2581 scope.go:117] "RemoveContainer" containerID="39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a"
Sep 18 18:59:31.733535 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:31.733500    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:31.733636 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:31.733594    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:31.733729 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:31.733644    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:32.745680 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:32.745637    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" event={"ID":"403a3ed9-8a91-4ca1-828b-64c0ca0741a9","Type":"ContainerStarted","Data":"cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81"}
Sep 18 18:59:32.749214 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:32.749182    2581 generic.go:356] "Generic (PLEG): container finished" podID="115a52cd-9845-4bf3-8577-557fbda14e55" containerID="25d979c60200ed7f16e2bb5d15830b5ae452f728468543b7a27cb94bd526d236" exitCode=0
Sep 18 18:59:32.749531 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:32.749259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" event={"ID":"115a52cd-9845-4bf3-8577-557fbda14e55","Type":"ContainerDied","Data":"25d979c60200ed7f16e2bb5d15830b5ae452f728468543b7a27cb94bd526d236"}
Sep 18 18:59:32.750047 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:32.750011    2581 scope.go:117] "RemoveContainer" containerID="25d979c60200ed7f16e2bb5d15830b5ae452f728468543b7a27cb94bd526d236"
Sep 18 18:59:32.751580 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:32.751534    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:32.751687 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:32.751636    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:32.751777 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:32.751684    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:59:33.581269 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:33.581230    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:59:33.581459 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:33.581339    2581 patch_prober.go:28] interesting pod/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 18 18:59:33.581459 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:33.581377    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 18 18:59:33.764688 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:33.764641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" event={"ID":"115a52cd-9845-4bf3-8577-557fbda14e55","Type":"ContainerStarted","Data":"1ddde02fdeb6dfa31dcb819f04ebc0c231c295a9b7566030045329237bf361dc"}
Sep 18 18:59:34.484168 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:34.484114    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:34.485162 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:34.485117    2581 patch_prober.go:28] interesting pod/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 18 18:59:34.485311 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:34.485204    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 18 18:59:34.583734 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:34.583382    2581 patch_prober.go:28] interesting pod/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 18 18:59:34.583734 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:34.583480    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 18 18:59:35.484554 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:35.484519    2581 patch_prober.go:28] interesting pod/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 18 18:59:35.485115 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:35.485086    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 18 18:59:35.581359 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:35.581321    2581 patch_prober.go:28] interesting pod/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 18 18:59:35.581570 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:35.581397    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 18 18:59:36.484473 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:36.484434    2581 patch_prober.go:28] interesting pod/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 18 18:59:36.484670 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:36.484504    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 18 18:59:36.545295 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:36.545252    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:36.581728 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:36.581676    2581 patch_prober.go:28] interesting pod/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 18 18:59:36.581946 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:36.581774    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 18 18:59:37.484823 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:37.484779    2581 patch_prober.go:28] interesting pod/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 18 18:59:37.485204 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:37.484836    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 18 18:59:37.581650 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:37.581611    2581 patch_prober.go:28] interesting pod/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 18 18:59:37.581974 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:37.581948    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 18 18:59:38.484313 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:38.484266    2581 patch_prober.go:28] interesting pod/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 18 18:59:38.484545 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:38.484335    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 18 18:59:38.586051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:38.586019    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:59:38.587005 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:38.586499    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:59:38.587472 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:38.587453    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 18:59:39.488834 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:39.488799    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:39.489833 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:39.489815    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:45.634450 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:45.634398    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"]
Sep 18 18:59:45.634992 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:45.634646    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy" containerID="cri-o://1ddde02fdeb6dfa31dcb819f04ebc0c231c295a9b7566030045329237bf361dc" gracePeriod=30
Sep 18 18:59:50.878336 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.878303    2581 generic.go:356] "Generic (PLEG): container finished" podID="115a52cd-9845-4bf3-8577-557fbda14e55" containerID="1ddde02fdeb6dfa31dcb819f04ebc0c231c295a9b7566030045329237bf361dc" exitCode=0
Sep 18 18:59:50.878848 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.878378    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" event={"ID":"115a52cd-9845-4bf3-8577-557fbda14e55","Type":"ContainerDied","Data":"1ddde02fdeb6dfa31dcb819f04ebc0c231c295a9b7566030045329237bf361dc"}
Sep 18 18:59:50.878848 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.878465    2581 scope.go:117] "RemoveContainer" containerID="25d979c60200ed7f16e2bb5d15830b5ae452f728468543b7a27cb94bd526d236"
Sep 18 18:59:50.902381 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.902347    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:50.985649 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.985606    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-istio-token\") pod \"115a52cd-9845-4bf3-8577-557fbda14e55\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") "
Sep 18 18:59:50.985849 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.985677    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-envoy\") pod \"115a52cd-9845-4bf3-8577-557fbda14e55\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") "
Sep 18 18:59:50.985849 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.985738    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115a52cd-9845-4bf3-8577-557fbda14e55-istiod-ca-cert\") pod \"115a52cd-9845-4bf3-8577-557fbda14e55\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") "
Sep 18 18:59:50.985849 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.985778    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-credential-socket\") pod \"115a52cd-9845-4bf3-8577-557fbda14e55\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") "
Sep 18 18:59:50.985849 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.985820    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-socket\") pod \"115a52cd-9845-4bf3-8577-557fbda14e55\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") "
Sep 18 18:59:50.985849 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.985845    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-data\") pod \"115a52cd-9845-4bf3-8577-557fbda14e55\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") "
Sep 18 18:59:50.986142 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.985926    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-certs\") pod \"115a52cd-9845-4bf3-8577-557fbda14e55\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") "
Sep 18 18:59:50.986142 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.985961    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmj8v\" (UniqueName: \"kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-kube-api-access-nmj8v\") pod \"115a52cd-9845-4bf3-8577-557fbda14e55\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") "
Sep 18 18:59:50.986142 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.986013    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115a52cd-9845-4bf3-8577-557fbda14e55-istio-podinfo\") pod \"115a52cd-9845-4bf3-8577-557fbda14e55\" (UID: \"115a52cd-9845-4bf3-8577-557fbda14e55\") "
Sep 18 18:59:50.986142 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.986103    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "115a52cd-9845-4bf3-8577-557fbda14e55" (UID: "115a52cd-9845-4bf3-8577-557fbda14e55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:59:50.986373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.986330    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "115a52cd-9845-4bf3-8577-557fbda14e55" (UID: "115a52cd-9845-4bf3-8577-557fbda14e55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:59:50.986373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.986344    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:59:50.986373 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.986344    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/115a52cd-9845-4bf3-8577-557fbda14e55-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "115a52cd-9845-4bf3-8577-557fbda14e55" (UID: "115a52cd-9845-4bf3-8577-557fbda14e55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:59:50.986546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.986467    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "115a52cd-9845-4bf3-8577-557fbda14e55" (UID: "115a52cd-9845-4bf3-8577-557fbda14e55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:59:50.986546 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.986482    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-data" (OuterVolumeSpecName: "istio-data") pod "115a52cd-9845-4bf3-8577-557fbda14e55" (UID: "115a52cd-9845-4bf3-8577-557fbda14e55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:59:50.988872 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.988650    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-istio-token" (OuterVolumeSpecName: "istio-token") pod "115a52cd-9845-4bf3-8577-557fbda14e55" (UID: "115a52cd-9845-4bf3-8577-557fbda14e55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:59:50.988872 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.988779    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "115a52cd-9845-4bf3-8577-557fbda14e55" (UID: "115a52cd-9845-4bf3-8577-557fbda14e55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:59:50.988872 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.988812    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/115a52cd-9845-4bf3-8577-557fbda14e55-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "115a52cd-9845-4bf3-8577-557fbda14e55" (UID: "115a52cd-9845-4bf3-8577-557fbda14e55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:59:50.988872 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:50.988847    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-kube-api-access-nmj8v" (OuterVolumeSpecName: "kube-api-access-nmj8v") pod "115a52cd-9845-4bf3-8577-557fbda14e55" (UID: "115a52cd-9845-4bf3-8577-557fbda14e55"). InnerVolumeSpecName "kube-api-access-nmj8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:59:51.087912 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.087871    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmj8v\" (UniqueName: \"kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-kube-api-access-nmj8v\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:59:51.087912 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.087905    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115a52cd-9845-4bf3-8577-557fbda14e55-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:59:51.087912 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.087915    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115a52cd-9845-4bf3-8577-557fbda14e55-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:59:51.087912 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.087923    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:59:51.088192 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.087931    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115a52cd-9845-4bf3-8577-557fbda14e55-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:59:51.088192 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.087939    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:59:51.088192 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.087946    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:59:51.088192 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.087953    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115a52cd-9845-4bf3-8577-557fbda14e55-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 18:59:51.885998 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.885963    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"
Sep 18 18:59:51.886487 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.886012    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47" event={"ID":"115a52cd-9845-4bf3-8577-557fbda14e55","Type":"ContainerDied","Data":"62298dfe1bdee6e87fc6d6e201d6c5a36092d04a2a9263c0b648badad0813ff9"}
Sep 18 18:59:51.886487 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.886068    2581 scope.go:117] "RemoveContainer" containerID="1ddde02fdeb6dfa31dcb819f04ebc0c231c295a9b7566030045329237bf361dc"
Sep 18 18:59:51.979374 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.979336    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"]
Sep 18 18:59:51.991290 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:51.991251    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8cbeacf3-openshift-default-557475794b-tfr47"]
Sep 18 18:59:52.319064 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:52.315125    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" path="/var/lib/kubelet/pods/115a52cd-9845-4bf3-8577-557fbda14e55/volumes"
Sep 18 18:59:55.832642 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.832610    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"]
Sep 18 18:59:55.833051 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.833037    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy"
Sep 18 18:59:55.833095 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.833053    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy"
Sep 18 18:59:55.833095 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.833073    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy"
Sep 18 18:59:55.833095 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.833078    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy"
Sep 18 18:59:55.833193 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.833148    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy"
Sep 18 18:59:55.836493 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.836470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:55.859092 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.859066    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-f4465858-serving-cert\""
Sep 18 18:59:55.879625 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.879592    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"]
Sep 18 18:59:55.932503 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.932461    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxcps\" (UniqueName: \"kubernetes.io/projected/46421654-7046-4cd1-a6dd-36ed17c4545a-kube-api-access-kxcps\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:55.932734 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.932519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46421654-7046-4cd1-a6dd-36ed17c4545a-openshift-service-ca\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:55.932734 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.932650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-maas-api-tls\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:55.932734 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:55.932677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-metrics-certs\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:56.033994 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.033954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-maas-api-tls\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:56.034178 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.034005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-metrics-certs\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:56.034178 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.034050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxcps\" (UniqueName: \"kubernetes.io/projected/46421654-7046-4cd1-a6dd-36ed17c4545a-kube-api-access-kxcps\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:56.034178 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.034102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46421654-7046-4cd1-a6dd-36ed17c4545a-openshift-service-ca\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:56.034905 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.034879    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46421654-7046-4cd1-a6dd-36ed17c4545a-openshift-service-ca\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:56.036619 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.036594    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-maas-api-tls\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:56.036751 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.036736    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-metrics-certs\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:56.048329 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.048301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxcps\" (UniqueName: \"kubernetes.io/projected/46421654-7046-4cd1-a6dd-36ed17c4545a-kube-api-access-kxcps\") pod \"maas-api-e2e-migrate-f4465858-67799dd799-48wwc\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:56.147638 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.147535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:56.448679 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.448645    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"]
Sep 18 18:59:56.449035 ip-10-0-136-52 kubenswrapper[2581]: W0918 18:59:56.449006    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46421654_7046_4cd1_a6dd_36ed17c4545a.slice/crio-53f502dd3b112fc28e5146aa0c1dfe00a9131e415f39ce1e27679d91bd12b03a WatchSource:0}: Error finding container 53f502dd3b112fc28e5146aa0c1dfe00a9131e415f39ce1e27679d91bd12b03a: Status 404 returned error can't find the container with id 53f502dd3b112fc28e5146aa0c1dfe00a9131e415f39ce1e27679d91bd12b03a
Sep 18 18:59:56.917163 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:56.917130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc" event={"ID":"46421654-7046-4cd1-a6dd-36ed17c4545a","Type":"ContainerStarted","Data":"53f502dd3b112fc28e5146aa0c1dfe00a9131e415f39ce1e27679d91bd12b03a"}
Sep 18 18:59:57.928471 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:57.928422    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc" event={"ID":"46421654-7046-4cd1-a6dd-36ed17c4545a","Type":"ContainerStarted","Data":"c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865"}
Sep 18 18:59:57.928995 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:57.928608    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 18:59:57.970299 ip-10-0-136-52 kubenswrapper[2581]: I0918 18:59:57.970240    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc" podStartSLOduration=2.361157079 podStartE2EDuration="2.970219702s" totalImagesPullingTime="609.062623ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:57.967446732 +0000 UTC m=+934.186454311" watchObservedRunningTime="2026-09-18 18:59:57.970219702 +0000 UTC m=+934.189227310"
Sep 18 19:00:03.941508 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:03.941480    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 19:00:31.357457 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.357419    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"]
Sep 18 19:00:31.357966 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.357949    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="115a52cd-9845-4bf3-8577-557fbda14e55" containerName="istio-proxy"
Sep 18 19:00:31.362416 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.362390    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.379641 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.379609    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-fb8843\"/\"default-dockercfg-hbhwf\""
Sep 18 19:00:31.379944 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.379612    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-fb8843\"/\"autores-model-fb8843-kserve-self-signed-certs\""
Sep 18 19:00:31.379944 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.379625    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-fb8843\"/\"kube-root-ca.crt\""
Sep 18 19:00:31.380259 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.380240    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-fb8843\"/\"openshift-service-ca.crt\""
Sep 18 19:00:31.394305 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.394269    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"]
Sep 18 19:00:31.476457 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.476421    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d48eadba-65ac-445d-adee-566cf0472754-tls-certs\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.476648 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.476473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-tmp-dir\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.476648 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.476497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-model-cache\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.476648 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.476518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjczd\" (UniqueName: \"kubernetes.io/projected/d48eadba-65ac-445d-adee-566cf0472754-kube-api-access-kjczd\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.476648 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.476542    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-home\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.476648 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.476561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-dshm\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.578178 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.578141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d48eadba-65ac-445d-adee-566cf0472754-tls-certs\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.578380 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.578205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-tmp-dir\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.578380 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.578233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-model-cache\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.578380 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.578258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjczd\" (UniqueName: \"kubernetes.io/projected/d48eadba-65ac-445d-adee-566cf0472754-kube-api-access-kjczd\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.578380 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.578293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-home\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.578380 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.578315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-dshm\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.578722 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.578674    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-tmp-dir\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.578816 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.578737    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-home\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.578816 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.578784    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-model-cache\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.580844 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.580819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-dshm\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.580983 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.580968    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d48eadba-65ac-445d-adee-566cf0472754-tls-certs\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.594010 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.593976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjczd\" (UniqueName: \"kubernetes.io/projected/d48eadba-65ac-445d-adee-566cf0472754-kube-api-access-kjczd\") pod \"autores-model-fb8843-kserve-5fc7dcbb95-4pc76\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") " pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.672833 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.672793    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:31.977647 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:31.977621    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"]
Sep 18 19:00:31.979897 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:00:31.979869    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd48eadba_65ac_445d_adee_566cf0472754.slice/crio-dddca42b3f58ab58588cd5584a44afb04497503990b94d4211047292bb9f4078 WatchSource:0}: Error finding container dddca42b3f58ab58588cd5584a44afb04497503990b94d4211047292bb9f4078: Status 404 returned error can't find the container with id dddca42b3f58ab58588cd5584a44afb04497503990b94d4211047292bb9f4078
Sep 18 19:00:32.138504 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:32.138463    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76" event={"ID":"d48eadba-65ac-445d-adee-566cf0472754","Type":"ContainerStarted","Data":"cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd"}
Sep 18 19:00:32.138504 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:32.138511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76" event={"ID":"d48eadba-65ac-445d-adee-566cf0472754","Type":"ContainerStarted","Data":"dddca42b3f58ab58588cd5584a44afb04497503990b94d4211047292bb9f4078"}
Sep 18 19:00:32.180585 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:32.180529    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76" podStartSLOduration=1.18051433 podStartE2EDuration="1.18051433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:32.180340934 +0000 UTC m=+968.399348514" watchObservedRunningTime="2026-09-18 19:00:32.18051433 +0000 UTC m=+968.399521908"
Sep 18 19:00:35.854281 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:35.854239    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"]
Sep 18 19:00:35.854749 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:35.854449    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc" podUID="46421654-7046-4cd1-a6dd-36ed17c4545a" containerName="maas-api" containerID="cri-o://c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865" gracePeriod=30
Sep 18 19:00:36.121371 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.121302    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 19:00:36.165945 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.165911    2581 generic.go:356] "Generic (PLEG): container finished" podID="46421654-7046-4cd1-a6dd-36ed17c4545a" containerID="c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865" exitCode=0
Sep 18 19:00:36.166121 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.165960    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"
Sep 18 19:00:36.166121 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.165998    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc" event={"ID":"46421654-7046-4cd1-a6dd-36ed17c4545a","Type":"ContainerDied","Data":"c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865"}
Sep 18 19:00:36.166121 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.166052    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc" event={"ID":"46421654-7046-4cd1-a6dd-36ed17c4545a","Type":"ContainerDied","Data":"53f502dd3b112fc28e5146aa0c1dfe00a9131e415f39ce1e27679d91bd12b03a"}
Sep 18 19:00:36.166121 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.166077    2581 scope.go:117] "RemoveContainer" containerID="c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865"
Sep 18 19:00:36.176463 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.176432    2581 scope.go:117] "RemoveContainer" containerID="c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865"
Sep 18 19:00:36.176463 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:00:36.177017    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865\": container with ID starting with c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865 not found: ID does not exist" containerID="c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865"
Sep 18 19:00:36.176463 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.177055    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865"} err="failed to get container status \"c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865\": rpc error: code = NotFound desc = could not find container \"c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865\": container with ID starting with c6ec60b3e27b172f34efb8a64ffe6ddd05c4663f4083ee7efda77428d7bb1865 not found: ID does not exist"
Sep 18 19:00:36.227922 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.227875    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46421654-7046-4cd1-a6dd-36ed17c4545a-openshift-service-ca\") pod \"46421654-7046-4cd1-a6dd-36ed17c4545a\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") "
Sep 18 19:00:36.228118 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.227963    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-metrics-certs\") pod \"46421654-7046-4cd1-a6dd-36ed17c4545a\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") "
Sep 18 19:00:36.228118 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.228005    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxcps\" (UniqueName: \"kubernetes.io/projected/46421654-7046-4cd1-a6dd-36ed17c4545a-kube-api-access-kxcps\") pod \"46421654-7046-4cd1-a6dd-36ed17c4545a\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") "
Sep 18 19:00:36.228118 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.228041    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-maas-api-tls\") pod \"46421654-7046-4cd1-a6dd-36ed17c4545a\" (UID: \"46421654-7046-4cd1-a6dd-36ed17c4545a\") "
Sep 18 19:00:36.228296 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.228256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46421654-7046-4cd1-a6dd-36ed17c4545a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46421654-7046-4cd1-a6dd-36ed17c4545a" (UID: "46421654-7046-4cd1-a6dd-36ed17c4545a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:00:36.228401 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.228381    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46421654-7046-4cd1-a6dd-36ed17c4545a-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:00:36.230478 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.230416    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "46421654-7046-4cd1-a6dd-36ed17c4545a" (UID: "46421654-7046-4cd1-a6dd-36ed17c4545a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:00:36.230675 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.230654    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46421654-7046-4cd1-a6dd-36ed17c4545a-kube-api-access-kxcps" (OuterVolumeSpecName: "kube-api-access-kxcps") pod "46421654-7046-4cd1-a6dd-36ed17c4545a" (UID: "46421654-7046-4cd1-a6dd-36ed17c4545a"). InnerVolumeSpecName "kube-api-access-kxcps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:00:36.230845 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.230820    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46421654-7046-4cd1-a6dd-36ed17c4545a" (UID: "46421654-7046-4cd1-a6dd-36ed17c4545a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:00:36.330030 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.330002    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:00:36.330030 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.330033    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxcps\" (UniqueName: \"kubernetes.io/projected/46421654-7046-4cd1-a6dd-36ed17c4545a-kube-api-access-kxcps\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:00:36.330274 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.330048    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46421654-7046-4cd1-a6dd-36ed17c4545a-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:00:36.508166 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.508118    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"]
Sep 18 19:00:36.519612 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:36.519572    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f4465858-67799dd799-48wwc"]
Sep 18 19:00:37.558625 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.558588    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"]
Sep 18 19:00:37.559064 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.559049    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46421654-7046-4cd1-a6dd-36ed17c4545a" containerName="maas-api"
Sep 18 19:00:37.559110 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.559066    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="46421654-7046-4cd1-a6dd-36ed17c4545a" containerName="maas-api"
Sep 18 19:00:37.559151 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.559141    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="46421654-7046-4cd1-a6dd-36ed17c4545a" containerName="maas-api"
Sep 18 19:00:37.563493 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.563470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.569955 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.569928    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-980e4a32-openshift-default-dockercfg-4wlvm\""
Sep 18 19:00:37.588413 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.588377    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"]
Sep 18 19:00:37.643966 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.643921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-credential-socket\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.643966 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.643965    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-podinfo\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.644228 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.644005    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qjkv\" (UniqueName: \"kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-kube-api-access-7qjkv\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.644228 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.644131    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istiod-ca-cert\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.644228 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.644177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-token\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.644228 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.644215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-socket\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.644408 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.644261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-data\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.644408 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.644303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-certs\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.644492 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.644431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-envoy\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.745359 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istiod-ca-cert\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.745359 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745359    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-token\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.745626 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-socket\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.745626 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745399    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-data\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.745626 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745427    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-certs\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.745626 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-envoy\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.745626 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-credential-socket\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.745626 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-podinfo\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.745626 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qjkv\" (UniqueName: \"kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-kube-api-access-7qjkv\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.746018 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745952    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-certs\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.746018 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.745988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-data\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.746089 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.746019    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-socket\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.746089 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.746065    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-credential-socket\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.746504 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.746484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istiod-ca-cert\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.748052 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.748018    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-envoy\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.748348 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.748325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-podinfo\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.759006 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.758976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qjkv\" (UniqueName: \"kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-kube-api-access-7qjkv\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.759155 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.758985    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-token\") pod \"e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") " pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:37.874275 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:37.874174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:38.188757 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:38.188732    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"]
Sep 18 19:00:38.190834 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:00:38.190804    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d3e0d0_e567_462f_9d7d_9da44e8949d3.slice/crio-b554569c4de4e4aef1f4a22c85746b48dae65ce8224a4fea726e1a76ae21a3a7 WatchSource:0}: Error finding container b554569c4de4e4aef1f4a22c85746b48dae65ce8224a4fea726e1a76ae21a3a7: Status 404 returned error can't find the container with id b554569c4de4e4aef1f4a22c85746b48dae65ce8224a4fea726e1a76ae21a3a7
Sep 18 19:00:38.193336 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:38.193306    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:00:38.193400 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:38.193374    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:00:38.193436 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:38.193403    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:00:38.315603 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:38.315570    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46421654-7046-4cd1-a6dd-36ed17c4545a" path="/var/lib/kubelet/pods/46421654-7046-4cd1-a6dd-36ed17c4545a/volumes"
Sep 18 19:00:39.188152 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:39.188117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" event={"ID":"f3d3e0d0-e567-462f-9d7d-9da44e8949d3","Type":"ContainerStarted","Data":"9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea"}
Sep 18 19:00:39.188666 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:39.188180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" event={"ID":"f3d3e0d0-e567-462f-9d7d-9da44e8949d3","Type":"ContainerStarted","Data":"b554569c4de4e4aef1f4a22c85746b48dae65ce8224a4fea726e1a76ae21a3a7"}
Sep 18 19:00:39.874697 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:39.874654    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:39.880567 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:39.880537    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:39.925394 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:39.925331    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" podStartSLOduration=2.925314132 podStartE2EDuration="2.925314132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:39.238324209 +0000 UTC m=+975.457331788" watchObservedRunningTime="2026-09-18 19:00:39.925314132 +0000 UTC m=+976.144321710"
Sep 18 19:00:40.193973 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:40.193943    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:40.195064 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:40.195041    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:00:41.673003 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:41.672963    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:41.673437 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:41.673068    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:41.686275 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:41.686245    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:42.216663 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:42.216632    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:00:45.418415 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.418382    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"]
Sep 18 19:00:45.423115 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.423090    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.427803 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.427758    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-980e4a32-serving-cert\""
Sep 18 19:00:45.439008 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.438973    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"]
Sep 18 19:00:45.525137 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.525088    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-maas-api-tls\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.525346 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.525169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081f35a8-dbe2-4c31-9da7-d723fdd717f0-openshift-service-ca\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.525346 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.525303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phw6b\" (UniqueName: \"kubernetes.io/projected/081f35a8-dbe2-4c31-9da7-d723fdd717f0-kube-api-access-phw6b\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.525435 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.525386    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-metrics-certs\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.626456 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.626411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-maas-api-tls\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.626780 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.626473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081f35a8-dbe2-4c31-9da7-d723fdd717f0-openshift-service-ca\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.626780 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.626526    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phw6b\" (UniqueName: \"kubernetes.io/projected/081f35a8-dbe2-4c31-9da7-d723fdd717f0-kube-api-access-phw6b\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.626780 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.626570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-metrics-certs\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.627495 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.627465    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081f35a8-dbe2-4c31-9da7-d723fdd717f0-openshift-service-ca\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.630287 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.630255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-metrics-certs\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.630632 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.630611    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-maas-api-tls\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.637138 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.637104    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phw6b\" (UniqueName: \"kubernetes.io/projected/081f35a8-dbe2-4c31-9da7-d723fdd717f0-kube-api-access-phw6b\") pod \"maas-api-e2e-ait-980e4a32-5d85fc556b-p578f\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:45.738083 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:45.737983    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:46.032373 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:46.032339    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"]
Sep 18 19:00:46.033477 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:00:46.033450    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081f35a8_dbe2_4c31_9da7_d723fdd717f0.slice/crio-c05f69728da4a34a40e61b9685107b77cc556e086a37bbd49aede3b031945215 WatchSource:0}: Error finding container c05f69728da4a34a40e61b9685107b77cc556e086a37bbd49aede3b031945215: Status 404 returned error can't find the container with id c05f69728da4a34a40e61b9685107b77cc556e086a37bbd49aede3b031945215
Sep 18 19:00:46.034882 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:46.034862    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:00:46.231389 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:46.231350    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f" event={"ID":"081f35a8-dbe2-4c31-9da7-d723fdd717f0","Type":"ContainerStarted","Data":"c05f69728da4a34a40e61b9685107b77cc556e086a37bbd49aede3b031945215"}
Sep 18 19:00:47.240076 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:47.240030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f" event={"ID":"081f35a8-dbe2-4c31-9da7-d723fdd717f0","Type":"ContainerStarted","Data":"87a842bc7e71337aceeacf5ae7801e589e743dfd6d90dc2028bdd321e0aa6741"}
Sep 18 19:00:47.240514 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:47.240109    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:00:47.291718 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:47.291658    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f" podStartSLOduration=1.8136232190000001 podStartE2EDuration="2.291642665s" totalImagesPullingTime="478.019446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:47.291370765 +0000 UTC m=+983.510378344" watchObservedRunningTime="2026-09-18 19:00:47.291642665 +0000 UTC m=+983.510650242"
Sep 18 19:00:53.252897 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:00:53.252860    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:01:02.333068 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:02.333026    2581 generic.go:356] "Generic (PLEG): container finished" podID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerID="9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea" exitCode=0
Sep 18 19:01:02.333515 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:02.333093    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" event={"ID":"f3d3e0d0-e567-462f-9d7d-9da44e8949d3","Type":"ContainerDied","Data":"9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea"}
Sep 18 19:01:02.333572 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:02.333547    2581 scope.go:117] "RemoveContainer" containerID="9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea"
Sep 18 19:01:02.334865 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:02.334834    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:01:02.335126 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:02.334907    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:01:02.335126 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:02.334942    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:01:03.346345 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:03.346306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" event={"ID":"f3d3e0d0-e567-462f-9d7d-9da44e8949d3","Type":"ContainerStarted","Data":"8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0"}
Sep 18 19:01:03.875371 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:03.875322    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:01:03.875610 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:03.875586    2581 patch_prober.go:28] interesting pod/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 18 19:01:03.875692 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:03.875645    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 18 19:01:04.875534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:04.875497    2581 patch_prober.go:28] interesting pod/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 18 19:01:04.876018 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:04.875555    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 18 19:01:05.875397 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:05.875353    2581 patch_prober.go:28] interesting pod/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 18 19:01:05.875615 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:05.875432    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 18 19:01:06.874567 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:06.874534    2581 patch_prober.go:28] interesting pod/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 18 19:01:06.874776 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:06.874602    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 18 19:01:07.875088 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:07.875045    2581 patch_prober.go:28] interesting pod/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 18 19:01:07.875507 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:07.875116    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 18 19:01:08.879187 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:08.879153    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:01:08.879572 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:08.879443    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:01:08.880444 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:08.880427    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:01:20.314603 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.314573    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"]
Sep 18 19:01:20.315081 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.314796    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9" podUID="9de0f90e-7420-4853-ba27-bf9b433ea545" containerName="maas-api" containerID="cri-o://4e395356b0d6318c3087bb5546b1cd4034c8f777b0bd0ee5db87774d31594f28" gracePeriod=30
Sep 18 19:01:20.466556 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.466146    2581 generic.go:356] "Generic (PLEG): container finished" podID="9de0f90e-7420-4853-ba27-bf9b433ea545" containerID="4e395356b0d6318c3087bb5546b1cd4034c8f777b0bd0ee5db87774d31594f28" exitCode=0
Sep 18 19:01:20.466556 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.466361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9" event={"ID":"9de0f90e-7420-4853-ba27-bf9b433ea545","Type":"ContainerDied","Data":"4e395356b0d6318c3087bb5546b1cd4034c8f777b0bd0ee5db87774d31594f28"}
Sep 18 19:01:20.584405 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.584375    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 19:01:20.702496 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.702460    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de0f90e-7420-4853-ba27-bf9b433ea545-openshift-service-ca\") pod \"9de0f90e-7420-4853-ba27-bf9b433ea545\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") "
Sep 18 19:01:20.702496 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.702497    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfxq2\" (UniqueName: \"kubernetes.io/projected/9de0f90e-7420-4853-ba27-bf9b433ea545-kube-api-access-hfxq2\") pod \"9de0f90e-7420-4853-ba27-bf9b433ea545\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") "
Sep 18 19:01:20.702791 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.702588    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-metrics-certs\") pod \"9de0f90e-7420-4853-ba27-bf9b433ea545\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") "
Sep 18 19:01:20.702791 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.702636    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-maas-api-tls\") pod \"9de0f90e-7420-4853-ba27-bf9b433ea545\" (UID: \"9de0f90e-7420-4853-ba27-bf9b433ea545\") "
Sep 18 19:01:20.702976 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.702945    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9de0f90e-7420-4853-ba27-bf9b433ea545-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9de0f90e-7420-4853-ba27-bf9b433ea545" (UID: "9de0f90e-7420-4853-ba27-bf9b433ea545"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:01:20.704938 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.704899    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9de0f90e-7420-4853-ba27-bf9b433ea545" (UID: "9de0f90e-7420-4853-ba27-bf9b433ea545"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:01:20.705066 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.704970    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9de0f90e-7420-4853-ba27-bf9b433ea545" (UID: "9de0f90e-7420-4853-ba27-bf9b433ea545"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:01:20.705066 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.705026    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9de0f90e-7420-4853-ba27-bf9b433ea545-kube-api-access-hfxq2" (OuterVolumeSpecName: "kube-api-access-hfxq2") pod "9de0f90e-7420-4853-ba27-bf9b433ea545" (UID: "9de0f90e-7420-4853-ba27-bf9b433ea545"). InnerVolumeSpecName "kube-api-access-hfxq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:01:20.803602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.803556    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de0f90e-7420-4853-ba27-bf9b433ea545-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:20.803602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.803592    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfxq2\" (UniqueName: \"kubernetes.io/projected/9de0f90e-7420-4853-ba27-bf9b433ea545-kube-api-access-hfxq2\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:20.803602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.803602    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:20.803602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:20.803610    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9de0f90e-7420-4853-ba27-bf9b433ea545-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:21.475003 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:21.474974    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"
Sep 18 19:01:21.475409 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:21.475021    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9" event={"ID":"9de0f90e-7420-4853-ba27-bf9b433ea545","Type":"ContainerDied","Data":"0f1b97e99ffcaa6cc0cdfb7c574f8ccaf93572632e3d6a4e56fcce865f6c500c"}
Sep 18 19:01:21.475409 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:21.475078    2581 scope.go:117] "RemoveContainer" containerID="4e395356b0d6318c3087bb5546b1cd4034c8f777b0bd0ee5db87774d31594f28"
Sep 18 19:01:21.635607 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:21.635573    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"]
Sep 18 19:01:21.674539 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:21.674507    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fb8843-6574ccbb84-9ghh9"]
Sep 18 19:01:22.314018 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:22.313984    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9de0f90e-7420-4853-ba27-bf9b433ea545" path="/var/lib/kubelet/pods/9de0f90e-7420-4853-ba27-bf9b433ea545/volumes"
Sep 18 19:01:26.326632 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:26.326592    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"]
Sep 18 19:01:29.316126 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.316089    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"]
Sep 18 19:01:29.316638 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.316329    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f" podUID="081f35a8-dbe2-4c31-9da7-d723fdd717f0" containerName="maas-api" containerID="cri-o://87a842bc7e71337aceeacf5ae7801e589e743dfd6d90dc2028bdd321e0aa6741" gracePeriod=30
Sep 18 19:01:29.533184 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.533153    2581 generic.go:356] "Generic (PLEG): container finished" podID="081f35a8-dbe2-4c31-9da7-d723fdd717f0" containerID="87a842bc7e71337aceeacf5ae7801e589e743dfd6d90dc2028bdd321e0aa6741" exitCode=0
Sep 18 19:01:29.533378 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.533228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f" event={"ID":"081f35a8-dbe2-4c31-9da7-d723fdd717f0","Type":"ContainerDied","Data":"87a842bc7e71337aceeacf5ae7801e589e743dfd6d90dc2028bdd321e0aa6741"}
Sep 18 19:01:29.590146 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.590111    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:01:29.697767 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.697722    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-maas-api-tls\") pod \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") "
Sep 18 19:01:29.697985 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.697802    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081f35a8-dbe2-4c31-9da7-d723fdd717f0-openshift-service-ca\") pod \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") "
Sep 18 19:01:29.697985 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.697850    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-metrics-certs\") pod \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") "
Sep 18 19:01:29.697985 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.697929    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phw6b\" (UniqueName: \"kubernetes.io/projected/081f35a8-dbe2-4c31-9da7-d723fdd717f0-kube-api-access-phw6b\") pod \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\" (UID: \"081f35a8-dbe2-4c31-9da7-d723fdd717f0\") "
Sep 18 19:01:29.698203 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.698172    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/081f35a8-dbe2-4c31-9da7-d723fdd717f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "081f35a8-dbe2-4c31-9da7-d723fdd717f0" (UID: "081f35a8-dbe2-4c31-9da7-d723fdd717f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:01:29.700245 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.700214    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "081f35a8-dbe2-4c31-9da7-d723fdd717f0" (UID: "081f35a8-dbe2-4c31-9da7-d723fdd717f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:01:29.700369 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.700257    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "081f35a8-dbe2-4c31-9da7-d723fdd717f0" (UID: "081f35a8-dbe2-4c31-9da7-d723fdd717f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:01:29.700369 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.700279    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081f35a8-dbe2-4c31-9da7-d723fdd717f0-kube-api-access-phw6b" (OuterVolumeSpecName: "kube-api-access-phw6b") pod "081f35a8-dbe2-4c31-9da7-d723fdd717f0" (UID: "081f35a8-dbe2-4c31-9da7-d723fdd717f0"). InnerVolumeSpecName "kube-api-access-phw6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:01:29.799428 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.799370    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:29.799733 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.799439    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081f35a8-dbe2-4c31-9da7-d723fdd717f0-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:29.799733 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.799457    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/081f35a8-dbe2-4c31-9da7-d723fdd717f0-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:29.799733 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:29.799471    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phw6b\" (UniqueName: \"kubernetes.io/projected/081f35a8-dbe2-4c31-9da7-d723fdd717f0-kube-api-access-phw6b\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:30.540918 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:30.540885    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"
Sep 18 19:01:30.540918 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:30.540897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f" event={"ID":"081f35a8-dbe2-4c31-9da7-d723fdd717f0","Type":"ContainerDied","Data":"c05f69728da4a34a40e61b9685107b77cc556e086a37bbd49aede3b031945215"}
Sep 18 19:01:30.541422 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:30.540965    2581 scope.go:117] "RemoveContainer" containerID="87a842bc7e71337aceeacf5ae7801e589e743dfd6d90dc2028bdd321e0aa6741"
Sep 18 19:01:30.592493 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:30.592391    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"]
Sep 18 19:01:30.609413 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:30.609379    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-980e4a32-5d85fc556b-p578f"]
Sep 18 19:01:31.243360 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:31.243319    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"]
Sep 18 19:01:31.243573 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:31.243546    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy" containerID="cri-o://8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0" gracePeriod=30
Sep 18 19:01:32.313207 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:32.313172    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="081f35a8-dbe2-4c31-9da7-d723fdd717f0" path="/var/lib/kubelet/pods/081f35a8-dbe2-4c31-9da7-d723fdd717f0/volumes"
Sep 18 19:01:36.505517 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.505436    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:01:36.563856 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.563819    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istiod-ca-cert\") pod \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") "
Sep 18 19:01:36.564057 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.563866    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-data\") pod \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") "
Sep 18 19:01:36.564057 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.563917    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-podinfo\") pod \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") "
Sep 18 19:01:36.564057 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.563961    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-envoy\") pod \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") "
Sep 18 19:01:36.564057 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.563986    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-socket\") pod \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") "
Sep 18 19:01:36.564057 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564038    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qjkv\" (UniqueName: \"kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-kube-api-access-7qjkv\") pod \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") "
Sep 18 19:01:36.564335 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564070    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-certs\") pod \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") "
Sep 18 19:01:36.564335 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564132    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-token\") pod \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") "
Sep 18 19:01:36.564335 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564169    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-credential-socket\") pod \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\" (UID: \"f3d3e0d0-e567-462f-9d7d-9da44e8949d3\") "
Sep 18 19:01:36.564485 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564317    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f3d3e0d0-e567-462f-9d7d-9da44e8949d3" (UID: "f3d3e0d0-e567-462f-9d7d-9da44e8949d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:01:36.564542 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564471    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f3d3e0d0-e567-462f-9d7d-9da44e8949d3" (UID: "f3d3e0d0-e567-462f-9d7d-9da44e8949d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:01:36.564542 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564497    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f3d3e0d0-e567-462f-9d7d-9da44e8949d3" (UID: "f3d3e0d0-e567-462f-9d7d-9da44e8949d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:01:36.564679 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564656    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "f3d3e0d0-e567-462f-9d7d-9da44e8949d3" (UID: "f3d3e0d0-e567-462f-9d7d-9da44e8949d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:01:36.564782 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564663    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:36.564782 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564671    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f3d3e0d0-e567-462f-9d7d-9da44e8949d3" (UID: "f3d3e0d0-e567-462f-9d7d-9da44e8949d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:01:36.564782 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564744    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:36.564782 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.564759    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:36.566487 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.566456    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f3d3e0d0-e567-462f-9d7d-9da44e8949d3" (UID: "f3d3e0d0-e567-462f-9d7d-9da44e8949d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:01:36.566634 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.566537    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f3d3e0d0-e567-462f-9d7d-9da44e8949d3" (UID: "f3d3e0d0-e567-462f-9d7d-9da44e8949d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:01:36.566634 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.566555    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-kube-api-access-7qjkv" (OuterVolumeSpecName: "kube-api-access-7qjkv") pod "f3d3e0d0-e567-462f-9d7d-9da44e8949d3" (UID: "f3d3e0d0-e567-462f-9d7d-9da44e8949d3"). InnerVolumeSpecName "kube-api-access-7qjkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:01:36.566754 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.566730    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "f3d3e0d0-e567-462f-9d7d-9da44e8949d3" (UID: "f3d3e0d0-e567-462f-9d7d-9da44e8949d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:01:36.581018 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.580986    2581 generic.go:356] "Generic (PLEG): container finished" podID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerID="8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0" exitCode=0
Sep 18 19:01:36.581213 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.581037    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"
Sep 18 19:01:36.581213 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.581071    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" event={"ID":"f3d3e0d0-e567-462f-9d7d-9da44e8949d3","Type":"ContainerDied","Data":"8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0"}
Sep 18 19:01:36.581213 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.581133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z" event={"ID":"f3d3e0d0-e567-462f-9d7d-9da44e8949d3","Type":"ContainerDied","Data":"b554569c4de4e4aef1f4a22c85746b48dae65ce8224a4fea726e1a76ae21a3a7"}
Sep 18 19:01:36.581213 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.581157    2581 scope.go:117] "RemoveContainer" containerID="8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0"
Sep 18 19:01:36.591474 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.591451    2581 scope.go:117] "RemoveContainer" containerID="9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea"
Sep 18 19:01:36.602868 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.602833    2581 scope.go:117] "RemoveContainer" containerID="8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0"
Sep 18 19:01:36.603261 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:01:36.603233    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0\": container with ID starting with 8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0 not found: ID does not exist" containerID="8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0"
Sep 18 19:01:36.603372 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.603267    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0"} err="failed to get container status \"8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0\": rpc error: code = NotFound desc = could not find container \"8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0\": container with ID starting with 8908f1ec5a573a3d9872aaf00b15ff026fbbc936599ba10b9a2048e52859d0b0 not found: ID does not exist"
Sep 18 19:01:36.603372 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.603285    2581 scope.go:117] "RemoveContainer" containerID="9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea"
Sep 18 19:01:36.603589 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:01:36.603572    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea\": container with ID starting with 9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea not found: ID does not exist" containerID="9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea"
Sep 18 19:01:36.603660 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.603592    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea"} err="failed to get container status \"9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea\": rpc error: code = NotFound desc = could not find container \"9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea\": container with ID starting with 9c0b01fc73312f6441495035a2a2664dfddb9e0d960e3ca9f2cdf44bc07543ea not found: ID does not exist"
Sep 18 19:01:36.624215 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.624174    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"]
Sep 18 19:01:36.637331 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.637291    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-980e4a32-openshift-default-58785f7db8-xvg7z"]
Sep 18 19:01:36.665440 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.665402    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:36.665440 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.665431    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:36.665440 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.665440    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qjkv\" (UniqueName: \"kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-kube-api-access-7qjkv\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:36.665440 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.665450    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:36.665753 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.665461    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:36.665753 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:36.665469    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3d3e0d0-e567-462f-9d7d-9da44e8949d3-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:38.314045 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:38.314009    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" path="/var/lib/kubelet/pods/f3d3e0d0-e567-462f-9d7d-9da44e8949d3/volumes"
Sep 18 19:01:41.347073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.347007    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76" podUID="d48eadba-65ac-445d-adee-566cf0472754" containerName="main" containerID="cri-o://cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd" gracePeriod=45
Sep 18 19:01:41.610899 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.610827    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:01:41.612597 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.612564    2581 generic.go:356] "Generic (PLEG): container finished" podID="d48eadba-65ac-445d-adee-566cf0472754" containerID="cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd" exitCode=0
Sep 18 19:01:41.612754 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.612609    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"
Sep 18 19:01:41.612754 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.612636    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76" event={"ID":"d48eadba-65ac-445d-adee-566cf0472754","Type":"ContainerDied","Data":"cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd"}
Sep 18 19:01:41.612754 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.612678    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76" event={"ID":"d48eadba-65ac-445d-adee-566cf0472754","Type":"ContainerDied","Data":"dddca42b3f58ab58588cd5584a44afb04497503990b94d4211047292bb9f4078"}
Sep 18 19:01:41.612754 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.612713    2581 scope.go:117] "RemoveContainer" containerID="cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd"
Sep 18 19:01:41.621789 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.621766    2581 scope.go:117] "RemoveContainer" containerID="cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd"
Sep 18 19:01:41.622143 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:01:41.622124    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd\": container with ID starting with cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd not found: ID does not exist" containerID="cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd"
Sep 18 19:01:41.622209 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.622156    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd"} err="failed to get container status \"cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd\": rpc error: code = NotFound desc = could not find container \"cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd\": container with ID starting with cc315f609b2d44333469b8a24134fb09e479aff22ce5e73245c08a00d4ad32dd not found: ID does not exist"
Sep 18 19:01:41.713771 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.713729    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d48eadba-65ac-445d-adee-566cf0472754-tls-certs\") pod \"d48eadba-65ac-445d-adee-566cf0472754\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") "
Sep 18 19:01:41.713971 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.713852    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-model-cache\") pod \"d48eadba-65ac-445d-adee-566cf0472754\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") "
Sep 18 19:01:41.713971 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.713872    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-dshm\") pod \"d48eadba-65ac-445d-adee-566cf0472754\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") "
Sep 18 19:01:41.713971 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.713899    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-home\") pod \"d48eadba-65ac-445d-adee-566cf0472754\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") "
Sep 18 19:01:41.713971 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.713930    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-tmp-dir\") pod \"d48eadba-65ac-445d-adee-566cf0472754\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") "
Sep 18 19:01:41.713971 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.713949    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjczd\" (UniqueName: \"kubernetes.io/projected/d48eadba-65ac-445d-adee-566cf0472754-kube-api-access-kjczd\") pod \"d48eadba-65ac-445d-adee-566cf0472754\" (UID: \"d48eadba-65ac-445d-adee-566cf0472754\") "
Sep 18 19:01:41.714251 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.714205    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-model-cache" (OuterVolumeSpecName: "model-cache") pod "d48eadba-65ac-445d-adee-566cf0472754" (UID: "d48eadba-65ac-445d-adee-566cf0472754"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:01:41.714251 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.714200    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-home" (OuterVolumeSpecName: "home") pod "d48eadba-65ac-445d-adee-566cf0472754" (UID: "d48eadba-65ac-445d-adee-566cf0472754"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:01:41.714374 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.714249    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d48eadba-65ac-445d-adee-566cf0472754" (UID: "d48eadba-65ac-445d-adee-566cf0472754"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:01:41.716386 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.716353    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d48eadba-65ac-445d-adee-566cf0472754-kube-api-access-kjczd" (OuterVolumeSpecName: "kube-api-access-kjczd") pod "d48eadba-65ac-445d-adee-566cf0472754" (UID: "d48eadba-65ac-445d-adee-566cf0472754"). InnerVolumeSpecName "kube-api-access-kjczd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:01:41.716524 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.716399    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-dshm" (OuterVolumeSpecName: "dshm") pod "d48eadba-65ac-445d-adee-566cf0472754" (UID: "d48eadba-65ac-445d-adee-566cf0472754"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:01:41.716524 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.716451    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d48eadba-65ac-445d-adee-566cf0472754-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d48eadba-65ac-445d-adee-566cf0472754" (UID: "d48eadba-65ac-445d-adee-566cf0472754"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:01:41.815680 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.815640    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:41.815680 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.815673    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:41.815680 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.815682    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:41.815680 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.815690    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48eadba-65ac-445d-adee-566cf0472754-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:41.815680 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.815712    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjczd\" (UniqueName: \"kubernetes.io/projected/d48eadba-65ac-445d-adee-566cf0472754-kube-api-access-kjczd\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:41.815680 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.815722    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d48eadba-65ac-445d-adee-566cf0472754-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:01:41.836912 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.836874    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"]
Sep 18 19:01:41.837372 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837354    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy"
Sep 18 19:01:41.837430 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837378    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy"
Sep 18 19:01:41.837430 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837393    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d48eadba-65ac-445d-adee-566cf0472754" containerName="main"
Sep 18 19:01:41.837430 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837402    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d48eadba-65ac-445d-adee-566cf0472754" containerName="main"
Sep 18 19:01:41.837430 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837413    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081f35a8-dbe2-4c31-9da7-d723fdd717f0" containerName="maas-api"
Sep 18 19:01:41.837430 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837422    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="081f35a8-dbe2-4c31-9da7-d723fdd717f0" containerName="maas-api"
Sep 18 19:01:41.837601 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837434    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9de0f90e-7420-4853-ba27-bf9b433ea545" containerName="maas-api"
Sep 18 19:01:41.837601 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837439    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9de0f90e-7420-4853-ba27-bf9b433ea545" containerName="maas-api"
Sep 18 19:01:41.837601 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837518    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d48eadba-65ac-445d-adee-566cf0472754" containerName="main"
Sep 18 19:01:41.837601 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837528    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy"
Sep 18 19:01:41.837601 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837534    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy"
Sep 18 19:01:41.837601 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837540    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9de0f90e-7420-4853-ba27-bf9b433ea545" containerName="maas-api"
Sep 18 19:01:41.837601 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.837548    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="081f35a8-dbe2-4c31-9da7-d723fdd717f0" containerName="maas-api"
Sep 18 19:01:41.841928 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.841895    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:41.846578 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.846550    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-3341b20d-serving-cert\""
Sep 18 19:01:41.866091 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.865985    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"]
Sep 18 19:01:41.916829 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.916792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbk56\" (UniqueName: \"kubernetes.io/projected/404195ea-94e6-414c-9ea6-194b5230ff16-kube-api-access-qbk56\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:41.917052 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.916859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-metrics-certs\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:41.917052 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.916896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-maas-api-tls\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:41.917052 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.917014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404195ea-94e6-414c-9ea6-194b5230ff16-openshift-service-ca\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:41.952696 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.952660    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"]
Sep 18 19:01:41.964365 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:41.964327    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-fb8843/autores-model-fb8843-kserve-5fc7dcbb95-4pc76"]
Sep 18 19:01:42.018497 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.018449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404195ea-94e6-414c-9ea6-194b5230ff16-openshift-service-ca\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:42.018756 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.018517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbk56\" (UniqueName: \"kubernetes.io/projected/404195ea-94e6-414c-9ea6-194b5230ff16-kube-api-access-qbk56\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:42.018756 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.018572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-metrics-certs\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:42.018756 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.018609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-maas-api-tls\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:42.018900 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:01:42.018788    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-serving-cert: secret "maas-api-e2e-derive-3341b20d-serving-cert" not found
Sep 18 19:01:42.018900 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:01:42.018864    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-maas-api-tls podName:404195ea-94e6-414c-9ea6-194b5230ff16 nodeName:}" failed. No retries permitted until 2026-09-18 19:01:42.518842426 +0000 UTC m=+1038.737849985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-maas-api-tls") pod "maas-api-e2e-derive-3341b20d-86f745b498-swqd5" (UID: "404195ea-94e6-414c-9ea6-194b5230ff16") : secret "maas-api-e2e-derive-3341b20d-serving-cert" not found
Sep 18 19:01:42.019197 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.019173    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404195ea-94e6-414c-9ea6-194b5230ff16-openshift-service-ca\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:42.021226 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.021206    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-metrics-certs\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:42.035372 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.035338    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbk56\" (UniqueName: \"kubernetes.io/projected/404195ea-94e6-414c-9ea6-194b5230ff16-kube-api-access-qbk56\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:42.314112 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.314076    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d48eadba-65ac-445d-adee-566cf0472754" path="/var/lib/kubelet/pods/d48eadba-65ac-445d-adee-566cf0472754/volumes"
Sep 18 19:01:42.524112 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.524069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-maas-api-tls\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:42.527108 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.527067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-maas-api-tls\") pod \"maas-api-e2e-derive-3341b20d-86f745b498-swqd5\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:42.752820 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:42.752783    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:43.091170 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:43.091142    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"]
Sep 18 19:01:43.092992 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:01:43.092953    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404195ea_94e6_414c_9ea6_194b5230ff16.slice/crio-44279fcea34d9c9b47e14c9e4526d60c67688270dc32da106d9243af552fd44e WatchSource:0}: Error finding container 44279fcea34d9c9b47e14c9e4526d60c67688270dc32da106d9243af552fd44e: Status 404 returned error can't find the container with id 44279fcea34d9c9b47e14c9e4526d60c67688270dc32da106d9243af552fd44e
Sep 18 19:01:43.628304 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:43.628269    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5" event={"ID":"404195ea-94e6-414c-9ea6-194b5230ff16","Type":"ContainerStarted","Data":"44279fcea34d9c9b47e14c9e4526d60c67688270dc32da106d9243af552fd44e"}
Sep 18 19:01:44.636354 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:44.636312    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5" event={"ID":"404195ea-94e6-414c-9ea6-194b5230ff16","Type":"ContainerStarted","Data":"f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099"}
Sep 18 19:01:44.636788 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:44.636418    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:01:44.711549 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:44.711478    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5" podStartSLOduration=3.243760816 podStartE2EDuration="3.711454431s" totalImagesPullingTime="467.693615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:01:44.708257245 +0000 UTC m=+1040.927264824" watchObservedRunningTime="2026-09-18 19:01:44.711454431 +0000 UTC m=+1040.930462012"
Sep 18 19:01:50.649295 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:01:50.649266    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:02:04.148884 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:04.148837    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"]
Sep 18 19:02:04.149401 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:04.149176    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy" containerID="cri-o://cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81" gracePeriod=30
Sep 18 19:02:05.022519 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.022483    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"]
Sep 18 19:02:05.022919 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.022907    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy"
Sep 18 19:02:05.022987 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.022921    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d3e0d0-e567-462f-9d7d-9da44e8949d3" containerName="istio-proxy"
Sep 18 19:02:05.026498 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.026466    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.029543 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.029518    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-cc7def-openshift-default-dockercfg-f8f5w\""
Sep 18 19:02:05.044372 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.044339    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"]
Sep 18 19:02:05.051997 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.051962    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d46f6e-7224-48c6-9af2-a2a18b158341-istiod-ca-cert\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.052176 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.052006    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-podinfo\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.052176 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.052054    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-certs\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.052176 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.052083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-data\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.052176 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.052103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mllfz\" (UniqueName: \"kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-kube-api-access-mllfz\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.052176 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.052167    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-socket\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.052347 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.052204    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-token\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.052347 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.052230    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-credential-socket\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.052347 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.052308    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-envoy\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.153049 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-credential-socket\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.153529 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153059    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-envoy\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.153529 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d46f6e-7224-48c6-9af2-a2a18b158341-istiod-ca-cert\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.153529 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-podinfo\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.153529 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153126    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-certs\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.153529 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-data\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.153529 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mllfz\" (UniqueName: \"kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-kube-api-access-mllfz\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.153529 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-socket\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.153529 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-token\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.154206 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153613    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-data\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.154206 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153655    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-certs\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.154206 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153826    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-credential-socket\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.154206 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-socket\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.154206 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.153917    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d46f6e-7224-48c6-9af2-a2a18b158341-istiod-ca-cert\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.155667 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.155639    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-envoy\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.155907 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.155889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-podinfo\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.162210 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.162177    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-token\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.162597 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.162573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mllfz\" (UniqueName: \"kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-kube-api-access-mllfz\") pod \"e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") " pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.338532 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.338439    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:05.622146 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.622097    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"]
Sep 18 19:02:05.624234 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:02:05.624199    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d46f6e_7224_48c6_9af2_a2a18b158341.slice/crio-5e13823b4f80291ccbe96507567797b055d6c75f563cb1390939a23bf9242fd1 WatchSource:0}: Error finding container 5e13823b4f80291ccbe96507567797b055d6c75f563cb1390939a23bf9242fd1: Status 404 returned error can't find the container with id 5e13823b4f80291ccbe96507567797b055d6c75f563cb1390939a23bf9242fd1
Sep 18 19:02:05.627463 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.627406    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:02:05.627600 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.627521    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:02:05.627600 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.627569    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:02:05.770286 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.770237    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" event={"ID":"43d46f6e-7224-48c6-9af2-a2a18b158341","Type":"ContainerStarted","Data":"051b0d1ccaa436f9334806ac195eae57abaee88c670ca241cb76d97b5efa4825"}
Sep 18 19:02:05.770286 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.770295    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" event={"ID":"43d46f6e-7224-48c6-9af2-a2a18b158341","Type":"ContainerStarted","Data":"5e13823b4f80291ccbe96507567797b055d6c75f563cb1390939a23bf9242fd1"}
Sep 18 19:02:05.800113 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:05.800059    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" podStartSLOduration=1.8000418059999999 podStartE2EDuration="1.800041806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:02:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:02:05.798938804 +0000 UTC m=+1062.017946383" watchObservedRunningTime="2026-09-18 19:02:05.800041806 +0000 UTC m=+1062.019049383"
Sep 18 19:02:06.177188 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.177154    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"]
Sep 18 19:02:06.177640 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.177386    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5" podUID="404195ea-94e6-414c-9ea6-194b5230ff16" containerName="maas-api" containerID="cri-o://f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099" gracePeriod=30
Sep 18 19:02:06.339219 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.339182    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:06.344902 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.344811    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:06.451611 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.451586    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:02:06.568269 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.568236    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404195ea-94e6-414c-9ea6-194b5230ff16-openshift-service-ca\") pod \"404195ea-94e6-414c-9ea6-194b5230ff16\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") "
Sep 18 19:02:06.568434 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.568278    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbk56\" (UniqueName: \"kubernetes.io/projected/404195ea-94e6-414c-9ea6-194b5230ff16-kube-api-access-qbk56\") pod \"404195ea-94e6-414c-9ea6-194b5230ff16\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") "
Sep 18 19:02:06.568434 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.568348    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-maas-api-tls\") pod \"404195ea-94e6-414c-9ea6-194b5230ff16\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") "
Sep 18 19:02:06.568434 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.568395    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-metrics-certs\") pod \"404195ea-94e6-414c-9ea6-194b5230ff16\" (UID: \"404195ea-94e6-414c-9ea6-194b5230ff16\") "
Sep 18 19:02:06.568650 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.568600    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/404195ea-94e6-414c-9ea6-194b5230ff16-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "404195ea-94e6-414c-9ea6-194b5230ff16" (UID: "404195ea-94e6-414c-9ea6-194b5230ff16"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:02:06.570788 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.570746    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/404195ea-94e6-414c-9ea6-194b5230ff16-kube-api-access-qbk56" (OuterVolumeSpecName: "kube-api-access-qbk56") pod "404195ea-94e6-414c-9ea6-194b5230ff16" (UID: "404195ea-94e6-414c-9ea6-194b5230ff16"). InnerVolumeSpecName "kube-api-access-qbk56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:06.570788 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.570763    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "404195ea-94e6-414c-9ea6-194b5230ff16" (UID: "404195ea-94e6-414c-9ea6-194b5230ff16"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:02:06.570937 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.570790    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "404195ea-94e6-414c-9ea6-194b5230ff16" (UID: "404195ea-94e6-414c-9ea6-194b5230ff16"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:02:06.669585 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.669546    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:06.669585 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.669578    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404195ea-94e6-414c-9ea6-194b5230ff16-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:06.669585 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.669588    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404195ea-94e6-414c-9ea6-194b5230ff16-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:06.669894 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.669599    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbk56\" (UniqueName: \"kubernetes.io/projected/404195ea-94e6-414c-9ea6-194b5230ff16-kube-api-access-qbk56\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:06.777052 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.776955    2581 generic.go:356] "Generic (PLEG): container finished" podID="404195ea-94e6-414c-9ea6-194b5230ff16" containerID="f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099" exitCode=0
Sep 18 19:02:06.777052 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.777017    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"
Sep 18 19:02:06.777252 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.777045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5" event={"ID":"404195ea-94e6-414c-9ea6-194b5230ff16","Type":"ContainerDied","Data":"f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099"}
Sep 18 19:02:06.777252 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.777132    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5" event={"ID":"404195ea-94e6-414c-9ea6-194b5230ff16","Type":"ContainerDied","Data":"44279fcea34d9c9b47e14c9e4526d60c67688270dc32da106d9243af552fd44e"}
Sep 18 19:02:06.777252 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.777156    2581 scope.go:117] "RemoveContainer" containerID="f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099"
Sep 18 19:02:06.777384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.777362    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:06.778646 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.778621    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:06.787831 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.787814    2581 scope.go:117] "RemoveContainer" containerID="f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099"
Sep 18 19:02:06.788162 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:02:06.788142    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099\": container with ID starting with f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099 not found: ID does not exist" containerID="f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099"
Sep 18 19:02:06.788234 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.788182    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099"} err="failed to get container status \"f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099\": rpc error: code = NotFound desc = could not find container \"f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099\": container with ID starting with f450c5b1de8e72b08398d0b5758b07aa216d0531552bd27e9ec29f655cff3099 not found: ID does not exist"
Sep 18 19:02:06.902939 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.902896    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"]
Sep 18 19:02:06.928892 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:06.928857    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3341b20d-86f745b498-swqd5"]
Sep 18 19:02:08.314450 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:08.314415    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="404195ea-94e6-414c-9ea6-194b5230ff16" path="/var/lib/kubelet/pods/404195ea-94e6-414c-9ea6-194b5230ff16/volumes"
Sep 18 19:02:08.588633 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:08.588544    2581 patch_prober.go:28] interesting pod/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 19:02:08.588633 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:08.588604    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 19:02:09.408527 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.408495    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 19:02:09.494375 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494341    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-envoy\") pod \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") "
Sep 18 19:02:09.494375 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494392    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-data\") pod \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") "
Sep 18 19:02:09.494594 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494431    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plj25\" (UniqueName: \"kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-kube-api-access-plj25\") pod \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") "
Sep 18 19:02:09.494594 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494471    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-credential-socket\") pod \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") "
Sep 18 19:02:09.494594 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494495    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-socket\") pod \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") "
Sep 18 19:02:09.494594 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494541    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-podinfo\") pod \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") "
Sep 18 19:02:09.494594 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494582    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istiod-ca-cert\") pod \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") "
Sep 18 19:02:09.494895 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494607    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-certs\") pod \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") "
Sep 18 19:02:09.494895 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494652    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-token\") pod \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\" (UID: \"403a3ed9-8a91-4ca1-828b-64c0ca0741a9\") "
Sep 18 19:02:09.494895 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494670    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "403a3ed9-8a91-4ca1-828b-64c0ca0741a9" (UID: "403a3ed9-8a91-4ca1-828b-64c0ca0741a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:09.494895 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494879    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "403a3ed9-8a91-4ca1-828b-64c0ca0741a9" (UID: "403a3ed9-8a91-4ca1-828b-64c0ca0741a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:09.495122 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.494986    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "403a3ed9-8a91-4ca1-828b-64c0ca0741a9" (UID: "403a3ed9-8a91-4ca1-828b-64c0ca0741a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:02:09.495122 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.495003    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "403a3ed9-8a91-4ca1-828b-64c0ca0741a9" (UID: "403a3ed9-8a91-4ca1-828b-64c0ca0741a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:09.495122 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.495064    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "403a3ed9-8a91-4ca1-828b-64c0ca0741a9" (UID: "403a3ed9-8a91-4ca1-828b-64c0ca0741a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:09.495122 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.495070    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:09.495122 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.495103    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:09.496992 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.496962    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "403a3ed9-8a91-4ca1-828b-64c0ca0741a9" (UID: "403a3ed9-8a91-4ca1-828b-64c0ca0741a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:09.497133 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.496990    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "403a3ed9-8a91-4ca1-828b-64c0ca0741a9" (UID: "403a3ed9-8a91-4ca1-828b-64c0ca0741a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:02:09.497390 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.497376    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "403a3ed9-8a91-4ca1-828b-64c0ca0741a9" (UID: "403a3ed9-8a91-4ca1-828b-64c0ca0741a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:09.497435 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.497409    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-kube-api-access-plj25" (OuterVolumeSpecName: "kube-api-access-plj25") pod "403a3ed9-8a91-4ca1-828b-64c0ca0741a9" (UID: "403a3ed9-8a91-4ca1-828b-64c0ca0741a9"). InnerVolumeSpecName "kube-api-access-plj25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:09.596019 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.595984    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:09.596019 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.596018    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:09.596019 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.596028    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:09.596019 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.596035    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:09.596300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.596044    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:09.596300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.596051    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:09.596300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.596059    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plj25\" (UniqueName: \"kubernetes.io/projected/403a3ed9-8a91-4ca1-828b-64c0ca0741a9-kube-api-access-plj25\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:02:09.799469 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.799373    2581 generic.go:356] "Generic (PLEG): container finished" podID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerID="cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81" exitCode=0
Sep 18 19:02:09.799469 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.799424    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"
Sep 18 19:02:09.799664 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.799467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" event={"ID":"403a3ed9-8a91-4ca1-828b-64c0ca0741a9","Type":"ContainerDied","Data":"cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81"}
Sep 18 19:02:09.799664 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.799524    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4" event={"ID":"403a3ed9-8a91-4ca1-828b-64c0ca0741a9","Type":"ContainerDied","Data":"e528ac51822b3a294935bc0a65e1d6720ebefc6d2388d19235d4f022343f0ce3"}
Sep 18 19:02:09.799664 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.799558    2581 scope.go:117] "RemoveContainer" containerID="cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81"
Sep 18 19:02:09.809802 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.809776    2581 scope.go:117] "RemoveContainer" containerID="39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a"
Sep 18 19:02:09.819104 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.819081    2581 scope.go:117] "RemoveContainer" containerID="cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81"
Sep 18 19:02:09.819486 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:02:09.819466    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81\": container with ID starting with cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81 not found: ID does not exist" containerID="cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81"
Sep 18 19:02:09.819534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.819496    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81"} err="failed to get container status \"cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81\": rpc error: code = NotFound desc = could not find container \"cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81\": container with ID starting with cecef65226d7a0bff2b0ae4f15a389bfc180caba00acab4f4c0f6d5c9ddc2c81 not found: ID does not exist"
Sep 18 19:02:09.819534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.819519    2581 scope.go:117] "RemoveContainer" containerID="39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a"
Sep 18 19:02:09.819850 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:02:09.819830    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a\": container with ID starting with 39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a not found: ID does not exist" containerID="39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a"
Sep 18 19:02:09.819928 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.819860    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a"} err="failed to get container status \"39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a\": rpc error: code = NotFound desc = could not find container \"39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a\": container with ID starting with 39363318902e6eb0d488e93f118501a8be0d43db3bed1a82741012590dc1f03a not found: ID does not exist"
Sep 18 19:02:09.834112 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.834078    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"]
Sep 18 19:02:09.847356 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:09.847315    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-fb8843-openshift-default-7b9bb99b6d-fxpd4"]
Sep 18 19:02:10.314018 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:10.313980    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" path="/var/lib/kubelet/pods/403a3ed9-8a91-4ca1-828b-64c0ca0741a9/volumes"
Sep 18 19:02:13.152385 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.152342    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"]
Sep 18 19:02:13.153024 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.153004    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="404195ea-94e6-414c-9ea6-194b5230ff16" containerName="maas-api"
Sep 18 19:02:13.153076 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.153030    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="404195ea-94e6-414c-9ea6-194b5230ff16" containerName="maas-api"
Sep 18 19:02:13.153076 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.153063    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy"
Sep 18 19:02:13.153076 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.153072    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy"
Sep 18 19:02:13.153242 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.153218    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy"
Sep 18 19:02:13.153242 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.153239    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="404195ea-94e6-414c-9ea6-194b5230ff16" containerName="maas-api"
Sep 18 19:02:13.153312 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.153251    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy"
Sep 18 19:02:13.158270 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.158239    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.163475 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.163450    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-cc7def-serving-cert\""
Sep 18 19:02:13.172144 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.172113    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"]
Sep 18 19:02:13.227163 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.227119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-metrics-certs\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.227382 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.227270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-maas-api-tls\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.227382 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.227303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckc79\" (UniqueName: \"kubernetes.io/projected/b14c4973-2feb-401c-aed2-738e15942a32-kube-api-access-ckc79\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.227382 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.227333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14c4973-2feb-401c-aed2-738e15942a32-openshift-service-ca\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.328411 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.328375    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14c4973-2feb-401c-aed2-738e15942a32-openshift-service-ca\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.328634 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.328441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-metrics-certs\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.328634 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.328561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-maas-api-tls\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.328634 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.328585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckc79\" (UniqueName: \"kubernetes.io/projected/b14c4973-2feb-401c-aed2-738e15942a32-kube-api-access-ckc79\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.329245 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.329216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14c4973-2feb-401c-aed2-738e15942a32-openshift-service-ca\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.332187 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.332151    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-maas-api-tls\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.332341 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.332215    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-metrics-certs\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.350991 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.350962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckc79\" (UniqueName: \"kubernetes.io/projected/b14c4973-2feb-401c-aed2-738e15942a32-kube-api-access-ckc79\") pod \"maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.470283 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.470240    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:13.756923 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.756887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"]
Sep 18 19:02:13.758232 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:02:13.758203    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14c4973_2feb_401c_aed2_738e15942a32.slice/crio-8853ba58cd948d48aff2331775360ff377e94ecc2e123caea94aefbd09d3be6c WatchSource:0}: Error finding container 8853ba58cd948d48aff2331775360ff377e94ecc2e123caea94aefbd09d3be6c: Status 404 returned error can't find the container with id 8853ba58cd948d48aff2331775360ff377e94ecc2e123caea94aefbd09d3be6c
Sep 18 19:02:13.830262 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:13.830211    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n" event={"ID":"b14c4973-2feb-401c-aed2-738e15942a32","Type":"ContainerStarted","Data":"8853ba58cd948d48aff2331775360ff377e94ecc2e123caea94aefbd09d3be6c"}
Sep 18 19:02:14.837465 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:14.837431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n" event={"ID":"b14c4973-2feb-401c-aed2-738e15942a32","Type":"ContainerStarted","Data":"c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc"}
Sep 18 19:02:14.837875 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:14.837580    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:14.867746 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:14.867656    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n" podStartSLOduration=1.425914256 podStartE2EDuration="1.867632117s" totalImagesPullingTime="441.717861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:02:14.865347031 +0000 UTC m=+1071.084354612" watchObservedRunningTime="2026-09-18 19:02:14.867632117 +0000 UTC m=+1071.086639696"
Sep 18 19:02:20.849629 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:20.849551    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:02:25.299838 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.299796    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"]
Sep 18 19:02:25.300674 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.300650    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy"
Sep 18 19:02:25.300834 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.300816    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="403a3ed9-8a91-4ca1-828b-64c0ca0741a9" containerName="istio-proxy"
Sep 18 19:02:25.304415 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.304389    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.308038 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.308015    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-942883\"/\"premium-w4-942883-kserve-self-signed-certs\""
Sep 18 19:02:25.308964 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.308941    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-942883\"/\"kube-root-ca.crt\""
Sep 18 19:02:25.309060 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.308949    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-942883\"/\"openshift-service-ca.crt\""
Sep 18 19:02:25.309129 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.309037    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-942883\"/\"default-dockercfg-v6rfk\""
Sep 18 19:02:25.320193 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.320158    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"]
Sep 18 19:02:25.458643 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.458599    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-model-cache\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.458643 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.458646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-dshm\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.458957 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.458688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tmp-dir\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.458957 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.458804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-home\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.458957 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.458839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tls-certs\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.458957 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.458914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76d8m\" (UniqueName: \"kubernetes.io/projected/d0ad3c77-9978-4fe7-845f-449a7846b7c9-kube-api-access-76d8m\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.560234 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.560127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-model-cache\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.560234 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.560179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-dshm\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.560234 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.560215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tmp-dir\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.560537 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.560254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-home\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.560537 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.560285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tls-certs\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.560537 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.560319    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76d8m\" (UniqueName: \"kubernetes.io/projected/d0ad3c77-9978-4fe7-845f-449a7846b7c9-kube-api-access-76d8m\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.560856 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.560827    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-model-cache\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.561032 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.560985    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tmp-dir\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.561362 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.561343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-home\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.563500 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.563475    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-dshm\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.563690 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.563667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tls-certs\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.570528 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.570462    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76d8m\" (UniqueName: \"kubernetes.io/projected/d0ad3c77-9978-4fe7-845f-449a7846b7c9-kube-api-access-76d8m\") pod \"premium-w4-942883-kserve-6df8d6fb7d-7fjm5\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") " pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.613659 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.613612    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:25.912122 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:25.912064    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"]
Sep 18 19:02:25.914459 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:02:25.914433    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0ad3c77_9978_4fe7_845f_449a7846b7c9.slice/crio-aea386b3c62b538685b69247113ea895d6282a4005e4e5853e9d3f716619bd7e WatchSource:0}: Error finding container aea386b3c62b538685b69247113ea895d6282a4005e4e5853e9d3f716619bd7e: Status 404 returned error can't find the container with id aea386b3c62b538685b69247113ea895d6282a4005e4e5853e9d3f716619bd7e
Sep 18 19:02:26.908061 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:26.908022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5" event={"ID":"d0ad3c77-9978-4fe7-845f-449a7846b7c9","Type":"ContainerStarted","Data":"c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3"}
Sep 18 19:02:26.908061 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:26.908066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5" event={"ID":"d0ad3c77-9978-4fe7-845f-449a7846b7c9","Type":"ContainerStarted","Data":"aea386b3c62b538685b69247113ea895d6282a4005e4e5853e9d3f716619bd7e"}
Sep 18 19:02:26.936189 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:26.936107    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5" podStartSLOduration=1.9360826599999998 podStartE2EDuration="1.93608266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:02:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:02:26.932669807 +0000 UTC m=+1083.151677379" watchObservedRunningTime="2026-09-18 19:02:26.93608266 +0000 UTC m=+1083.155090238"
Sep 18 19:02:33.951142 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:33.951107    2581 generic.go:356] "Generic (PLEG): container finished" podID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerID="051b0d1ccaa436f9334806ac195eae57abaee88c670ca241cb76d97b5efa4825" exitCode=0
Sep 18 19:02:33.951553 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:33.951188    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" event={"ID":"43d46f6e-7224-48c6-9af2-a2a18b158341","Type":"ContainerDied","Data":"051b0d1ccaa436f9334806ac195eae57abaee88c670ca241cb76d97b5efa4825"}
Sep 18 19:02:33.951674 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:33.951657    2581 scope.go:117] "RemoveContainer" containerID="051b0d1ccaa436f9334806ac195eae57abaee88c670ca241cb76d97b5efa4825"
Sep 18 19:02:33.952863 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:33.952835    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:02:33.952961 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:33.952903    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:02:33.952961 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:33.952934    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:02:34.961327 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:34.961292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" event={"ID":"43d46f6e-7224-48c6-9af2-a2a18b158341","Type":"ContainerStarted","Data":"b3144b3c22fe59b6abc7a36dc17b6cf6b6b138285f226b94c254c6d2ff9acb2d"}
Sep 18 19:02:35.340101 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:35.339997    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:35.340495 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:35.340470    2581 patch_prober.go:28] interesting pod/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:02:35.340688 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:35.340659    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 19:02:35.614588 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:35.614470    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:35.614588 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:35.614544    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:35.652005 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:35.651968    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:35.979852 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:35.979818    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:02:36.339862 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:36.339756    2581 patch_prober.go:28] interesting pod/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:02:36.340654 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:36.340208    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 19:02:36.778091 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:36.778044    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:37.339443 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:37.339404    2581 patch_prober.go:28] interesting pod/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:02:37.340100 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:37.340065    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 19:02:38.339132 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:38.339084    2581 patch_prober.go:28] interesting pod/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:02:38.339366 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:38.339157    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 19:02:39.339118 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:39.339074    2581 patch_prober.go:28] interesting pod/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:02:39.339842 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:39.339806    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 19:02:40.339346 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:40.339304    2581 patch_prober.go:28] interesting pod/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:02:40.339986 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:40.339951    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 19:02:41.344262 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:41.344230    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:41.345380 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:41.345361    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:02:50.922997 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:50.922959    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"]
Sep 18 19:02:50.926757 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:50.926692    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:50.933073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:50.933046    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3a81e5\"/\"test-model-3a81e5-kserve-self-signed-certs\""
Sep 18 19:02:50.934113 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:50.934088    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3a81e5\"/\"openshift-service-ca.crt\""
Sep 18 19:02:50.934375 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:50.934359    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3a81e5\"/\"kube-root-ca.crt\""
Sep 18 19:02:50.942757 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:50.942695    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"]
Sep 18 19:02:50.944374 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:50.944346    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3a81e5\"/\"default-dockercfg-5sc65\""
Sep 18 19:02:51.023639 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.023582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxjwp\" (UniqueName: \"kubernetes.io/projected/e6582d3e-2c2a-4c3d-acab-d0781368bad4-kube-api-access-sxjwp\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.023639 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.023642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-home\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.023936 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.023668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-dshm\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.023936 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.023810    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tmp-dir\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.023936 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.023855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-model-cache\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.023936 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.023916    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tls-certs\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.125478 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.125430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tmp-dir\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.125679 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.125487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-model-cache\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.125679 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.125567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tls-certs\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.125679 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.125639    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxjwp\" (UniqueName: \"kubernetes.io/projected/e6582d3e-2c2a-4c3d-acab-d0781368bad4-kube-api-access-sxjwp\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.125679 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.125664    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-home\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.126002 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.125695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-dshm\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.126002 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.125933    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tmp-dir\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.126172 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.126149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-model-cache\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.126293 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.126264    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-home\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.128397 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.128364    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-dshm\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.128781 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.128759    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tls-certs\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.141243 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.141207    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxjwp\" (UniqueName: \"kubernetes.io/projected/e6582d3e-2c2a-4c3d-acab-d0781368bad4-kube-api-access-sxjwp\") pod \"test-model-3a81e5-kserve-86c45b659-2ntf7\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") " pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.238317 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.238224    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:02:51.522889 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:51.522862    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"]
Sep 18 19:02:51.525505 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:02:51.525476    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6582d3e_2c2a_4c3d_acab_d0781368bad4.slice/crio-cbcf110c774a8a201e90bb185bf75cadcc837a73ba284576a9620ab706ba4e8c WatchSource:0}: Error finding container cbcf110c774a8a201e90bb185bf75cadcc837a73ba284576a9620ab706ba4e8c: Status 404 returned error can't find the container with id cbcf110c774a8a201e90bb185bf75cadcc837a73ba284576a9620ab706ba4e8c
Sep 18 19:02:52.067041 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:52.066997    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7" event={"ID":"e6582d3e-2c2a-4c3d-acab-d0781368bad4","Type":"ContainerStarted","Data":"f8dd8c6234403904281638a2c41572c297711f02f9af4dca65b93c5ac9a928eb"}
Sep 18 19:02:52.067041 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:52.067048    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7" event={"ID":"e6582d3e-2c2a-4c3d-acab-d0781368bad4","Type":"ContainerStarted","Data":"cbcf110c774a8a201e90bb185bf75cadcc837a73ba284576a9620ab706ba4e8c"}
Sep 18 19:02:52.088973 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:02:52.088918    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7" podStartSLOduration=2.088900609 podStartE2EDuration="2.088900609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:02:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:02:52.08827845 +0000 UTC m=+1108.307286033" watchObservedRunningTime="2026-09-18 19:02:52.088900609 +0000 UTC m=+1108.307908187"
Sep 18 19:03:01.239430 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:01.239388    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:03:01.239940 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:01.239451    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:03:01.252593 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:01.252561    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:03:02.135834 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:02.135804    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:03:39.102093 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.102048    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"]
Sep 18 19:03:39.106824 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.106787    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.111631 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.111594    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-cc7def\"/\"openshift-service-ca.crt\""
Sep 18 19:03:39.112921 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.112897    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-cc7def\"/\"kube-root-ca.crt\""
Sep 18 19:03:39.113079 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.112941    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-cc7def\"/\"default-dockercfg-p9v4c\""
Sep 18 19:03:39.113079 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.112941    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-cc7def\"/\"explicit-model-cc7def-kserve-self-signed-certs\""
Sep 18 19:03:39.125797 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.125765    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"]
Sep 18 19:03:39.195962 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.195924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-model-cache\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.196169 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.195989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76da854e-c8d5-4d0e-b785-914e8d510837-tls-certs\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.196169 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.196064    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-home\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.196169 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.196093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-tmp-dir\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.196307 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.196190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmcsc\" (UniqueName: \"kubernetes.io/projected/76da854e-c8d5-4d0e-b785-914e8d510837-kube-api-access-mmcsc\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.196307 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.196239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-dshm\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.297365 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.297321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-dshm\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.297567 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.297375    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-model-cache\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.297567 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.297411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76da854e-c8d5-4d0e-b785-914e8d510837-tls-certs\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.297567 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.297429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-home\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.297809 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.297579    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-tmp-dir\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.297809 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.297723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmcsc\" (UniqueName: \"kubernetes.io/projected/76da854e-c8d5-4d0e-b785-914e8d510837-kube-api-access-mmcsc\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.297919 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.297866    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-home\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.297968 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.297937    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-tmp-dir\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.298500 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.298475    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-model-cache\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.299952 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.299927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-dshm\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.300236 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.300215    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76da854e-c8d5-4d0e-b785-914e8d510837-tls-certs\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.329622 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.329506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmcsc\" (UniqueName: \"kubernetes.io/projected/76da854e-c8d5-4d0e-b785-914e8d510837-kube-api-access-mmcsc\") pod \"explicit-model-cc7def-kserve-59b7bd5fdf-pskn5\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") " pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.417760 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.417722    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:39.722405 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:39.722373    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"]
Sep 18 19:03:39.725138 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:03:39.725109    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76da854e_c8d5_4d0e_b785_914e8d510837.slice/crio-589f0fc4795bb3bb60ddbf4c9a54caae019dbb549fd318f8b7a98167d19582d9 WatchSource:0}: Error finding container 589f0fc4795bb3bb60ddbf4c9a54caae019dbb549fd318f8b7a98167d19582d9: Status 404 returned error can't find the container with id 589f0fc4795bb3bb60ddbf4c9a54caae019dbb549fd318f8b7a98167d19582d9
Sep 18 19:03:40.358726 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:40.358660    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5" event={"ID":"76da854e-c8d5-4d0e-b785-914e8d510837","Type":"ContainerStarted","Data":"9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6"}
Sep 18 19:03:40.359233 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:40.358787    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5" event={"ID":"76da854e-c8d5-4d0e-b785-914e8d510837","Type":"ContainerStarted","Data":"589f0fc4795bb3bb60ddbf4c9a54caae019dbb549fd318f8b7a98167d19582d9"}
Sep 18 19:03:40.388321 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:40.388252    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5" podStartSLOduration=1.388230493 podStartE2EDuration="1.388230493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:03:40.383470023 +0000 UTC m=+1156.602477603" watchObservedRunningTime="2026-09-18 19:03:40.388230493 +0000 UTC m=+1156.607238118"
Sep 18 19:03:49.418128 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:49.418083    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:49.418650 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:49.418170    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:49.430940 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:49.430910    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:03:50.435877 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:03:50.435845    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:04:01.734623 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.734591    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"]
Sep 18 19:04:01.738797 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.738758    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.741582 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.741558    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fa6711c6-serving-cert\""
Sep 18 19:04:01.750928 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.750897    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"]
Sep 18 19:04:01.838606 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.838562    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-openshift-service-ca\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.838606 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.838612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzqds\" (UniqueName: \"kubernetes.io/projected/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-kube-api-access-mzqds\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.838870 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.838630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-maas-api-tls\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.838870 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.838652    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-metrics-certs\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.940190 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.940151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-openshift-service-ca\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.940190 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.940198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzqds\" (UniqueName: \"kubernetes.io/projected/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-kube-api-access-mzqds\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.940475 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.940219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-maas-api-tls\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.940475 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.940241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-metrics-certs\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.940933 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.940900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-openshift-service-ca\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.942989 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.942967    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-maas-api-tls\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.943088 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.942993    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-metrics-certs\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:01.951296 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:01.951260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzqds\" (UniqueName: \"kubernetes.io/projected/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-kube-api-access-mzqds\") pod \"maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:02.054603 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:02.054506    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:02.350624 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:04:02.350526    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod362d98b2_40f2_4ff2_84b9_d80d6707c5c4.slice/crio-bce7b7945016492bdbfa1c4c206d4ac75573faaefe4b0fcb62aa5c0e070b7541 WatchSource:0}: Error finding container bce7b7945016492bdbfa1c4c206d4ac75573faaefe4b0fcb62aa5c0e070b7541: Status 404 returned error can't find the container with id bce7b7945016492bdbfa1c4c206d4ac75573faaefe4b0fcb62aa5c0e070b7541
Sep 18 19:04:02.351989 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:02.351860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"]
Sep 18 19:04:02.502481 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:02.502444    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l" event={"ID":"362d98b2-40f2-4ff2-84b9-d80d6707c5c4","Type":"ContainerStarted","Data":"bce7b7945016492bdbfa1c4c206d4ac75573faaefe4b0fcb62aa5c0e070b7541"}
Sep 18 19:04:03.510685 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:03.510646    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l" event={"ID":"362d98b2-40f2-4ff2-84b9-d80d6707c5c4","Type":"ContainerStarted","Data":"abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919"}
Sep 18 19:04:03.511132 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:03.510763    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:09.524011 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:09.523972    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:09.553159 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:09.553094    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l" podStartSLOduration=8.003159879 podStartE2EDuration="8.553071661s" totalImagesPullingTime="549.911782ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:04:03.542012242 +0000 UTC m=+1179.761019819" watchObservedRunningTime="2026-09-18 19:04:09.553071661 +0000 UTC m=+1185.772079239"
Sep 18 19:04:27.821612 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:27.821574    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"]
Sep 18 19:04:27.822239 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:27.821904    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n" podUID="b14c4973-2feb-401c-aed2-738e15942a32" containerName="maas-api" containerID="cri-o://c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc" gracePeriod=30
Sep 18 19:04:28.103884 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.103848    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:04:28.212328 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.212272    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckc79\" (UniqueName: \"kubernetes.io/projected/b14c4973-2feb-401c-aed2-738e15942a32-kube-api-access-ckc79\") pod \"b14c4973-2feb-401c-aed2-738e15942a32\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") "
Sep 18 19:04:28.212328 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.212334    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14c4973-2feb-401c-aed2-738e15942a32-openshift-service-ca\") pod \"b14c4973-2feb-401c-aed2-738e15942a32\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") "
Sep 18 19:04:28.212572 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.212435    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-metrics-certs\") pod \"b14c4973-2feb-401c-aed2-738e15942a32\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") "
Sep 18 19:04:28.212572 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.212540    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-maas-api-tls\") pod \"b14c4973-2feb-401c-aed2-738e15942a32\" (UID: \"b14c4973-2feb-401c-aed2-738e15942a32\") "
Sep 18 19:04:28.212660 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.212638    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b14c4973-2feb-401c-aed2-738e15942a32-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b14c4973-2feb-401c-aed2-738e15942a32" (UID: "b14c4973-2feb-401c-aed2-738e15942a32"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:04:28.212930 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.212912    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b14c4973-2feb-401c-aed2-738e15942a32-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:28.214788 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.214760    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b14c4973-2feb-401c-aed2-738e15942a32" (UID: "b14c4973-2feb-401c-aed2-738e15942a32"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:04:28.214889 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.214784    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b14c4973-2feb-401c-aed2-738e15942a32" (UID: "b14c4973-2feb-401c-aed2-738e15942a32"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:04:28.214889 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.214812    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b14c4973-2feb-401c-aed2-738e15942a32-kube-api-access-ckc79" (OuterVolumeSpecName: "kube-api-access-ckc79") pod "b14c4973-2feb-401c-aed2-738e15942a32" (UID: "b14c4973-2feb-401c-aed2-738e15942a32"). InnerVolumeSpecName "kube-api-access-ckc79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:04:28.314225 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.314191    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:28.314225 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.314218    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b14c4973-2feb-401c-aed2-738e15942a32-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:28.314225 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.314227    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckc79\" (UniqueName: \"kubernetes.io/projected/b14c4973-2feb-401c-aed2-738e15942a32-kube-api-access-ckc79\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:28.670245 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.670212    2581 generic.go:356] "Generic (PLEG): container finished" podID="b14c4973-2feb-401c-aed2-738e15942a32" containerID="c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc" exitCode=0
Sep 18 19:04:28.670472 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.670272    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"
Sep 18 19:04:28.670472 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.670293    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n" event={"ID":"b14c4973-2feb-401c-aed2-738e15942a32","Type":"ContainerDied","Data":"c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc"}
Sep 18 19:04:28.670472 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.670366    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n" event={"ID":"b14c4973-2feb-401c-aed2-738e15942a32","Type":"ContainerDied","Data":"8853ba58cd948d48aff2331775360ff377e94ecc2e123caea94aefbd09d3be6c"}
Sep 18 19:04:28.670472 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.670392    2581 scope.go:117] "RemoveContainer" containerID="c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc"
Sep 18 19:04:28.682949 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.682904    2581 scope.go:117] "RemoveContainer" containerID="c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc"
Sep 18 19:04:28.683599 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:04:28.683570    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc\": container with ID starting with c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc not found: ID does not exist" containerID="c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc"
Sep 18 19:04:28.683771 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.683607    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc"} err="failed to get container status \"c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc\": rpc error: code = NotFound desc = could not find container \"c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc\": container with ID starting with c36331324a36d9a202f54575a746348368f2636dc71e703843c173de763183bc not found: ID does not exist"
Sep 18 19:04:28.701004 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.700971    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"]
Sep 18 19:04:28.707791 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:28.707754    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-cc7def-7f4cd47568-mrt5n"]
Sep 18 19:04:30.314925 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:30.314885    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b14c4973-2feb-401c-aed2-738e15942a32" path="/var/lib/kubelet/pods/b14c4973-2feb-401c-aed2-738e15942a32/volumes"
Sep 18 19:04:33.708290 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:33.708245    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"]
Sep 18 19:04:41.801963 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:41.801917    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"]
Sep 18 19:04:41.802419 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:41.802208    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l" podUID="362d98b2-40f2-4ff2-84b9-d80d6707c5c4" containerName="maas-api" containerID="cri-o://abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919" gracePeriod=30
Sep 18 19:04:42.074437 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.074403    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:42.161395 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.161326    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzqds\" (UniqueName: \"kubernetes.io/projected/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-kube-api-access-mzqds\") pod \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") "
Sep 18 19:04:42.161619 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.161410    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-metrics-certs\") pod \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") "
Sep 18 19:04:42.161619 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.161463    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-openshift-service-ca\") pod \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") "
Sep 18 19:04:42.161619 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.161490    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-maas-api-tls\") pod \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\" (UID: \"362d98b2-40f2-4ff2-84b9-d80d6707c5c4\") "
Sep 18 19:04:42.161939 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.161910    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "362d98b2-40f2-4ff2-84b9-d80d6707c5c4" (UID: "362d98b2-40f2-4ff2-84b9-d80d6707c5c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:04:42.163812 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.163779    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "362d98b2-40f2-4ff2-84b9-d80d6707c5c4" (UID: "362d98b2-40f2-4ff2-84b9-d80d6707c5c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:04:42.163929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.163831    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "362d98b2-40f2-4ff2-84b9-d80d6707c5c4" (UID: "362d98b2-40f2-4ff2-84b9-d80d6707c5c4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:04:42.163929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.163848    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-kube-api-access-mzqds" (OuterVolumeSpecName: "kube-api-access-mzqds") pod "362d98b2-40f2-4ff2-84b9-d80d6707c5c4" (UID: "362d98b2-40f2-4ff2-84b9-d80d6707c5c4"). InnerVolumeSpecName "kube-api-access-mzqds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:04:42.263198 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.263155    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzqds\" (UniqueName: \"kubernetes.io/projected/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-kube-api-access-mzqds\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:42.263198 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.263193    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:42.263198 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.263202    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:42.263451 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.263211    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/362d98b2-40f2-4ff2-84b9-d80d6707c5c4-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:42.763355 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.763317    2581 generic.go:356] "Generic (PLEG): container finished" podID="362d98b2-40f2-4ff2-84b9-d80d6707c5c4" containerID="abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919" exitCode=0
Sep 18 19:04:42.763355 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.763362    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"
Sep 18 19:04:42.763638 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.763377    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l" event={"ID":"362d98b2-40f2-4ff2-84b9-d80d6707c5c4","Type":"ContainerDied","Data":"abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919"}
Sep 18 19:04:42.763638 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.763426    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l" event={"ID":"362d98b2-40f2-4ff2-84b9-d80d6707c5c4","Type":"ContainerDied","Data":"bce7b7945016492bdbfa1c4c206d4ac75573faaefe4b0fcb62aa5c0e070b7541"}
Sep 18 19:04:42.763638 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.763450    2581 scope.go:117] "RemoveContainer" containerID="abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919"
Sep 18 19:04:42.774028 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.774000    2581 scope.go:117] "RemoveContainer" containerID="abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919"
Sep 18 19:04:42.774403 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:04:42.774375    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919\": container with ID starting with abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919 not found: ID does not exist" containerID="abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919"
Sep 18 19:04:42.774463 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.774414    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919"} err="failed to get container status \"abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919\": rpc error: code = NotFound desc = could not find container \"abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919\": container with ID starting with abae16b12c0e283c7376eccd5071867d7c27b184e5f5141dec82a5a009ca1919 not found: ID does not exist"
Sep 18 19:04:42.791850 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.791816    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"]
Sep 18 19:04:42.794740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:42.794691    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa6711c6-8577dcb8dd-j9r5l"]
Sep 18 19:04:44.314776 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:44.314745    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="362d98b2-40f2-4ff2-84b9-d80d6707c5c4" path="/var/lib/kubelet/pods/362d98b2-40f2-4ff2-84b9-d80d6707c5c4/volumes"
Sep 18 19:04:48.730136 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:48.730087    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5" podUID="76da854e-c8d5-4d0e-b785-914e8d510837" containerName="main" containerID="cri-o://9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6" gracePeriod=45
Sep 18 19:04:48.995225 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:48.995150    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:04:49.132947 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.132906    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-home\") pod \"76da854e-c8d5-4d0e-b785-914e8d510837\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") "
Sep 18 19:04:49.133160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.132982    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-model-cache\") pod \"76da854e-c8d5-4d0e-b785-914e8d510837\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") "
Sep 18 19:04:49.133160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.133019    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-tmp-dir\") pod \"76da854e-c8d5-4d0e-b785-914e8d510837\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") "
Sep 18 19:04:49.133160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.133049    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76da854e-c8d5-4d0e-b785-914e8d510837-tls-certs\") pod \"76da854e-c8d5-4d0e-b785-914e8d510837\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") "
Sep 18 19:04:49.133160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.133075    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-dshm\") pod \"76da854e-c8d5-4d0e-b785-914e8d510837\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") "
Sep 18 19:04:49.133160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.133106    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmcsc\" (UniqueName: \"kubernetes.io/projected/76da854e-c8d5-4d0e-b785-914e8d510837-kube-api-access-mmcsc\") pod \"76da854e-c8d5-4d0e-b785-914e8d510837\" (UID: \"76da854e-c8d5-4d0e-b785-914e8d510837\") "
Sep 18 19:04:49.133442 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.133304    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-home" (OuterVolumeSpecName: "home") pod "76da854e-c8d5-4d0e-b785-914e8d510837" (UID: "76da854e-c8d5-4d0e-b785-914e8d510837"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:49.133442 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.133320    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "76da854e-c8d5-4d0e-b785-914e8d510837" (UID: "76da854e-c8d5-4d0e-b785-914e8d510837"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:49.133442 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.133307    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-model-cache" (OuterVolumeSpecName: "model-cache") pod "76da854e-c8d5-4d0e-b785-914e8d510837" (UID: "76da854e-c8d5-4d0e-b785-914e8d510837"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:49.133442 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.133425    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:49.133442 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.133440    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:49.133666 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.133452    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:49.136059 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.136005    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76da854e-c8d5-4d0e-b785-914e8d510837-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "76da854e-c8d5-4d0e-b785-914e8d510837" (UID: "76da854e-c8d5-4d0e-b785-914e8d510837"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:04:49.136185 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.136087    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-dshm" (OuterVolumeSpecName: "dshm") pod "76da854e-c8d5-4d0e-b785-914e8d510837" (UID: "76da854e-c8d5-4d0e-b785-914e8d510837"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:49.136185 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.136095    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76da854e-c8d5-4d0e-b785-914e8d510837-kube-api-access-mmcsc" (OuterVolumeSpecName: "kube-api-access-mmcsc") pod "76da854e-c8d5-4d0e-b785-914e8d510837" (UID: "76da854e-c8d5-4d0e-b785-914e8d510837"). InnerVolumeSpecName "kube-api-access-mmcsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:04:49.234746 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.234689    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76da854e-c8d5-4d0e-b785-914e8d510837-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:49.234746 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.234737    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76da854e-c8d5-4d0e-b785-914e8d510837-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:49.234746 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.234746    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmcsc\" (UniqueName: \"kubernetes.io/projected/76da854e-c8d5-4d0e-b785-914e8d510837-kube-api-access-mmcsc\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:04:49.811843 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.811809    2581 generic.go:356] "Generic (PLEG): container finished" podID="76da854e-c8d5-4d0e-b785-914e8d510837" containerID="9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6" exitCode=0
Sep 18 19:04:49.812301 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.811854    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"
Sep 18 19:04:49.812301 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.811895    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5" event={"ID":"76da854e-c8d5-4d0e-b785-914e8d510837","Type":"ContainerDied","Data":"9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6"}
Sep 18 19:04:49.812301 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.811938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5" event={"ID":"76da854e-c8d5-4d0e-b785-914e8d510837","Type":"ContainerDied","Data":"589f0fc4795bb3bb60ddbf4c9a54caae019dbb549fd318f8b7a98167d19582d9"}
Sep 18 19:04:49.812301 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.811955    2581 scope.go:117] "RemoveContainer" containerID="9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6"
Sep 18 19:04:49.821992 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.821970    2581 scope.go:117] "RemoveContainer" containerID="9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6"
Sep 18 19:04:49.822350 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:04:49.822321    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6\": container with ID starting with 9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6 not found: ID does not exist" containerID="9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6"
Sep 18 19:04:49.822419 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.822365    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6"} err="failed to get container status \"9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6\": rpc error: code = NotFound desc = could not find container \"9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6\": container with ID starting with 9e2f2b8c177c6e2bba6f7ae6791f387f2e25b5c25ed6aca4b1e9a7747c63f9e6 not found: ID does not exist"
Sep 18 19:04:49.845652 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.845609    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"]
Sep 18 19:04:49.853186 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:49.853149    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-cc7def/explicit-model-cc7def-kserve-59b7bd5fdf-pskn5"]
Sep 18 19:04:50.317053 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:50.317018    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76da854e-c8d5-4d0e-b785-914e8d510837" path="/var/lib/kubelet/pods/76da854e-c8d5-4d0e-b785-914e8d510837/volumes"
Sep 18 19:04:58.977418 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.977376    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"]
Sep 18 19:04:58.978031 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.978010    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b14c4973-2feb-401c-aed2-738e15942a32" containerName="maas-api"
Sep 18 19:04:58.978031 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.978033    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b14c4973-2feb-401c-aed2-738e15942a32" containerName="maas-api"
Sep 18 19:04:58.978159 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.978044    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76da854e-c8d5-4d0e-b785-914e8d510837" containerName="main"
Sep 18 19:04:58.978159 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.978053    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="76da854e-c8d5-4d0e-b785-914e8d510837" containerName="main"
Sep 18 19:04:58.978159 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.978078    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="362d98b2-40f2-4ff2-84b9-d80d6707c5c4" containerName="maas-api"
Sep 18 19:04:58.978159 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.978086    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="362d98b2-40f2-4ff2-84b9-d80d6707c5c4" containerName="maas-api"
Sep 18 19:04:58.978371 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.978177    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b14c4973-2feb-401c-aed2-738e15942a32" containerName="maas-api"
Sep 18 19:04:58.978371 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.978189    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="76da854e-c8d5-4d0e-b785-914e8d510837" containerName="main"
Sep 18 19:04:58.978371 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.978196    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="362d98b2-40f2-4ff2-84b9-d80d6707c5c4" containerName="maas-api"
Sep 18 19:04:58.981454 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.981418    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:58.991809 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.991775    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-1a444c\"/\"kube-root-ca.crt\""
Sep 18 19:04:58.992092 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.992064    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1a444c\"/\"default-dockercfg-c8v6m\""
Sep 18 19:04:58.992092 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.992095    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1a444c\"/\"distinct-w0-1a444c-kserve-self-signed-certs\""
Sep 18 19:04:58.992342 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:58.992102    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-1a444c\"/\"openshift-service-ca.crt\""
Sep 18 19:04:59.013783 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.013748    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"]
Sep 18 19:04:59.022777 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.022739    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-home\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.022976 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.022808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tmp-dir\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.022976 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.022850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tls-certs\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.022976 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.022869    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-dshm\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.023120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.023060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngsrp\" (UniqueName: \"kubernetes.io/projected/74148608-cbd8-4cc6-a708-7a0bef72f7bb-kube-api-access-ngsrp\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.023120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.023109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-model-cache\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.124041 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.124005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-home\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.124041 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.124047    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tmp-dir\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.124286 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.124076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tls-certs\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.124286 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.124091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-dshm\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.124286 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.124137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngsrp\" (UniqueName: \"kubernetes.io/projected/74148608-cbd8-4cc6-a708-7a0bef72f7bb-kube-api-access-ngsrp\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.124286 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.124157    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-model-cache\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.124504 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.124482    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tmp-dir\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.124563 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.124524    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-home\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.124563 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.124545    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-model-cache\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.127218 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.127188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-dshm\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.127469 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.127447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tls-certs\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.135577 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.135542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngsrp\" (UniqueName: \"kubernetes.io/projected/74148608-cbd8-4cc6-a708-7a0bef72f7bb-kube-api-access-ngsrp\") pod \"distinct-w0-1a444c-kserve-7657c4789b-rtzf7\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.292094 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.291994    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:04:59.607286 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:04:59.607257    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74148608_cbd8_4cc6_a708_7a0bef72f7bb.slice/crio-8d9b30523308417629e25f1983c47e6067ce256f4d6ba2cb01b94cdd831d5dbe WatchSource:0}: Error finding container 8d9b30523308417629e25f1983c47e6067ce256f4d6ba2cb01b94cdd831d5dbe: Status 404 returned error can't find the container with id 8d9b30523308417629e25f1983c47e6067ce256f4d6ba2cb01b94cdd831d5dbe
Sep 18 19:04:59.612215 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.612183    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"]
Sep 18 19:04:59.877302 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.877200    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7" event={"ID":"74148608-cbd8-4cc6-a708-7a0bef72f7bb","Type":"ContainerStarted","Data":"e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04"}
Sep 18 19:04:59.877302 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.877250    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7" event={"ID":"74148608-cbd8-4cc6-a708-7a0bef72f7bb","Type":"ContainerStarted","Data":"8d9b30523308417629e25f1983c47e6067ce256f4d6ba2cb01b94cdd831d5dbe"}
Sep 18 19:04:59.916553 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:04:59.916487    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7" podStartSLOduration=1.916466662 podStartE2EDuration="1.916466662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:04:59.913008085 +0000 UTC m=+1236.132015663" watchObservedRunningTime="2026-09-18 19:04:59.916466662 +0000 UTC m=+1236.135474301"
Sep 18 19:05:09.293101 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:09.293052    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:05:09.293101 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:09.293106    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:05:09.306193 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:09.306155    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:05:09.949297 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:09.949265    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:05:13.763205 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:13.763161    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"]
Sep 18 19:05:13.763673 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:13.763386    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy" containerID="cri-o://b3144b3c22fe59b6abc7a36dc17b6cf6b6b138285f226b94c254c6d2ff9acb2d" gracePeriod=30
Sep 18 19:05:17.861403 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:17.861315    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"]
Sep 18 19:05:17.865195 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:17.865166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:17.868813 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:17.868786    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-3b5822-serving-cert\""
Sep 18 19:05:17.885743 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:17.885694    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"]
Sep 18 19:05:17.912786 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:17.912745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppphd\" (UniqueName: \"kubernetes.io/projected/d060d921-0a18-469c-847a-b9eb82aa5ae0-kube-api-access-ppphd\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:17.912967 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:17.912806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d060d921-0a18-469c-847a-b9eb82aa5ae0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:17.912967 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:17.912889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:17.913041 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:17.912967    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.014030 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.013988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.014193 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.014076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.014193 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:05:18.014126    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-serving-cert: secret "maas-api-e2e-shared-a-w2-3b5822-serving-cert" not found
Sep 18 19:05:18.014193 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.014148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppphd\" (UniqueName: \"kubernetes.io/projected/d060d921-0a18-469c-847a-b9eb82aa5ae0-kube-api-access-ppphd\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.014193 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:05:18.014188    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-maas-api-tls podName:d060d921-0a18-469c-847a-b9eb82aa5ae0 nodeName:}" failed. No retries permitted until 2026-09-18 19:05:18.514172794 +0000 UTC m=+1254.733180354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-maas-api-tls") pod "maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z" (UID: "d060d921-0a18-469c-847a-b9eb82aa5ae0") : secret "maas-api-e2e-shared-a-w2-3b5822-serving-cert" not found
Sep 18 19:05:18.014421 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.014222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d060d921-0a18-469c-847a-b9eb82aa5ae0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.014883 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.014855    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d060d921-0a18-469c-847a-b9eb82aa5ae0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.016852 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.016825    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.025816 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.025779    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppphd\" (UniqueName: \"kubernetes.io/projected/d060d921-0a18-469c-847a-b9eb82aa5ae0-kube-api-access-ppphd\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.520040 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.519996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.522802 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.522769    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.777246 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.777199    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:18.994179 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.994139    2581 generic.go:356] "Generic (PLEG): container finished" podID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerID="b3144b3c22fe59b6abc7a36dc17b6cf6b6b138285f226b94c254c6d2ff9acb2d" exitCode=0
Sep 18 19:05:18.994677 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.994181    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" event={"ID":"43d46f6e-7224-48c6-9af2-a2a18b158341","Type":"ContainerDied","Data":"b3144b3c22fe59b6abc7a36dc17b6cf6b6b138285f226b94c254c6d2ff9acb2d"}
Sep 18 19:05:18.994677 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:18.994256    2581 scope.go:117] "RemoveContainer" containerID="051b0d1ccaa436f9334806ac195eae57abaee88c670ca241cb76d97b5efa4825"
Sep 18 19:05:19.019267 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.019223    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:05:19.083888 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.083556    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"]
Sep 18 19:05:19.126944 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.126899    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-certs\") pod \"43d46f6e-7224-48c6-9af2-a2a18b158341\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") "
Sep 18 19:05:19.127126 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.126979    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-credential-socket\") pod \"43d46f6e-7224-48c6-9af2-a2a18b158341\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") "
Sep 18 19:05:19.127126 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127031    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-socket\") pod \"43d46f6e-7224-48c6-9af2-a2a18b158341\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") "
Sep 18 19:05:19.127126 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127059    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-token\") pod \"43d46f6e-7224-48c6-9af2-a2a18b158341\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") "
Sep 18 19:05:19.127126 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127113    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-data\") pod \"43d46f6e-7224-48c6-9af2-a2a18b158341\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") "
Sep 18 19:05:19.127363 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127141    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-envoy\") pod \"43d46f6e-7224-48c6-9af2-a2a18b158341\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") "
Sep 18 19:05:19.127363 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127185    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mllfz\" (UniqueName: \"kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-kube-api-access-mllfz\") pod \"43d46f6e-7224-48c6-9af2-a2a18b158341\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") "
Sep 18 19:05:19.127363 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127218    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-podinfo\") pod \"43d46f6e-7224-48c6-9af2-a2a18b158341\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") "
Sep 18 19:05:19.127363 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127243    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d46f6e-7224-48c6-9af2-a2a18b158341-istiod-ca-cert\") pod \"43d46f6e-7224-48c6-9af2-a2a18b158341\" (UID: \"43d46f6e-7224-48c6-9af2-a2a18b158341\") "
Sep 18 19:05:19.127363 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127285    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "43d46f6e-7224-48c6-9af2-a2a18b158341" (UID: "43d46f6e-7224-48c6-9af2-a2a18b158341"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:05:19.127622 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127515    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-data" (OuterVolumeSpecName: "istio-data") pod "43d46f6e-7224-48c6-9af2-a2a18b158341" (UID: "43d46f6e-7224-48c6-9af2-a2a18b158341"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:05:19.127673 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127631    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:05:19.127673 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127649    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:05:19.127809 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127773    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "43d46f6e-7224-48c6-9af2-a2a18b158341" (UID: "43d46f6e-7224-48c6-9af2-a2a18b158341"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:05:19.128012 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.127992    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "43d46f6e-7224-48c6-9af2-a2a18b158341" (UID: "43d46f6e-7224-48c6-9af2-a2a18b158341"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:05:19.128382 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.128352    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d46f6e-7224-48c6-9af2-a2a18b158341-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "43d46f6e-7224-48c6-9af2-a2a18b158341" (UID: "43d46f6e-7224-48c6-9af2-a2a18b158341"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:05:19.130431 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.130386    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "43d46f6e-7224-48c6-9af2-a2a18b158341" (UID: "43d46f6e-7224-48c6-9af2-a2a18b158341"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:05:19.130675 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.130647    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "43d46f6e-7224-48c6-9af2-a2a18b158341" (UID: "43d46f6e-7224-48c6-9af2-a2a18b158341"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:05:19.131267 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.131240    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-kube-api-access-mllfz" (OuterVolumeSpecName: "kube-api-access-mllfz") pod "43d46f6e-7224-48c6-9af2-a2a18b158341" (UID: "43d46f6e-7224-48c6-9af2-a2a18b158341"). InnerVolumeSpecName "kube-api-access-mllfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:19.131267 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.131251    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-token" (OuterVolumeSpecName: "istio-token") pod "43d46f6e-7224-48c6-9af2-a2a18b158341" (UID: "43d46f6e-7224-48c6-9af2-a2a18b158341"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:05:19.229019 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.228971    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:05:19.229019 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.229014    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mllfz\" (UniqueName: \"kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-kube-api-access-mllfz\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:05:19.229019 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.229030    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:05:19.229272 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.229045    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d46f6e-7224-48c6-9af2-a2a18b158341-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:05:19.229272 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.229057    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:05:19.229272 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.229071    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d46f6e-7224-48c6-9af2-a2a18b158341-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:05:19.229272 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:19.229084    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d46f6e-7224-48c6-9af2-a2a18b158341-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:05:20.002853 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:20.002810    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z" event={"ID":"d060d921-0a18-469c-847a-b9eb82aa5ae0","Type":"ContainerStarted","Data":"4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3"}
Sep 18 19:05:20.003394 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:20.002983    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:20.003394 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:20.003010    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z" event={"ID":"d060d921-0a18-469c-847a-b9eb82aa5ae0","Type":"ContainerStarted","Data":"b619482defe9553e83cf0d72c45295fbcc17e3390d70a73b827c5d254d889a94"}
Sep 18 19:05:20.004323 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:20.004299    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8" event={"ID":"43d46f6e-7224-48c6-9af2-a2a18b158341","Type":"ContainerDied","Data":"5e13823b4f80291ccbe96507567797b055d6c75f563cb1390939a23bf9242fd1"}
Sep 18 19:05:20.004464 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:20.004357    2581 scope.go:117] "RemoveContainer" containerID="b3144b3c22fe59b6abc7a36dc17b6cf6b6b138285f226b94c254c6d2ff9acb2d"
Sep 18 19:05:20.004464 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:20.004358    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"
Sep 18 19:05:20.033155 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:20.033064    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z" podStartSLOduration=2.266916632 podStartE2EDuration="3.033044362s" totalImagesPullingTime="766.12773ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:05:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:05:20.030693068 +0000 UTC m=+1256.249700660" watchObservedRunningTime="2026-09-18 19:05:20.033044362 +0000 UTC m=+1256.252051939"
Sep 18 19:05:20.054724 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:20.054662    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"]
Sep 18 19:05:20.059799 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:20.059751    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-cc7def-openshift-default-8497c9d7b8-422v8"]
Sep 18 19:05:20.314124 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:20.314090    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" path="/var/lib/kubelet/pods/43d46f6e-7224-48c6-9af2-a2a18b158341/volumes"
Sep 18 19:05:26.017753 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:26.017697    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:05:42.344967 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.344929    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"]
Sep 18 19:05:42.345788 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.345357    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy"
Sep 18 19:05:42.345788 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.345369    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy"
Sep 18 19:05:42.345788 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.345382    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy"
Sep 18 19:05:42.345788 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.345387    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy"
Sep 18 19:05:42.345788 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.345448    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy"
Sep 18 19:05:42.348914 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.348882    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.352888 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.352860    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-4805a7-openshift-default-dockercfg-vzmqk\""
Sep 18 19:05:42.360514 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.360482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-socket\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.360672 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.360533    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-credential-socket\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.360672 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.360577    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istiod-ca-cert\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.360672 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.360662    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-data\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.360859 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.360681    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-certs\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.360859 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.360724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qghgh\" (UniqueName: \"kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-kube-api-access-qghgh\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.360859 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.360760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-envoy\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.360859 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.360842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-podinfo\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.361023 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.360874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-token\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.376515 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.376478    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"]
Sep 18 19:05:42.462348 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-data\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.462553 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-certs\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.462553 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qghgh\" (UniqueName: \"kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-kube-api-access-qghgh\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.462553 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462418    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-envoy\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.462553 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-podinfo\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.462804 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462676    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-token\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.462864 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-socket\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.462864 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-data\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.462973 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462892    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-certs\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.462973 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-credential-socket\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.463081 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.462965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istiod-ca-cert\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.463179 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.463153    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-socket\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.463259 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.463233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-credential-socket\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.463542 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.463523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istiod-ca-cert\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.464936 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.464911    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-envoy\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.465155 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.465136    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-podinfo\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.471947 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.471920    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-token\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.472148 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.472126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qghgh\" (UniqueName: \"kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-kube-api-access-qghgh\") pod \"e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") " pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.659634 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.659527    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:42.947478 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.947434    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"]
Sep 18 19:05:42.952813 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:05:42.952780    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d458cd2_a72f_4c7d_99a6_aa40698fd016.slice/crio-05b05992f4af3fb3c56db6c510f22de029f479a0dfe8c51b255f0b65fa773ddb WatchSource:0}: Error finding container 05b05992f4af3fb3c56db6c510f22de029f479a0dfe8c51b255f0b65fa773ddb: Status 404 returned error can't find the container with id 05b05992f4af3fb3c56db6c510f22de029f479a0dfe8c51b255f0b65fa773ddb
Sep 18 19:05:42.955912 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.955876    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:05:42.956005 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.955971    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:05:42.956005 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:42.955999    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:05:43.155050 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:43.155013    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" event={"ID":"0d458cd2-a72f-4c7d-99a6-aa40698fd016","Type":"ContainerStarted","Data":"e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2"}
Sep 18 19:05:43.155236 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:43.155078    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" event={"ID":"0d458cd2-a72f-4c7d-99a6-aa40698fd016","Type":"ContainerStarted","Data":"05b05992f4af3fb3c56db6c510f22de029f479a0dfe8c51b255f0b65fa773ddb"}
Sep 18 19:05:43.188361 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:43.188273    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" podStartSLOduration=1.188246725 podStartE2EDuration="1.188246725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:05:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:05:43.183436087 +0000 UTC m=+1279.402443691" watchObservedRunningTime="2026-09-18 19:05:43.188246725 +0000 UTC m=+1279.407254304"
Sep 18 19:05:43.660210 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:43.660163    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:44.665560 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:44.665524    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:45.170829 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:45.170786    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:45.171796 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:45.171772    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:05:49.742896 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.742859    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"]
Sep 18 19:05:49.743360 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.743346    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d46f6e-7224-48c6-9af2-a2a18b158341" containerName="istio-proxy"
Sep 18 19:05:49.746794 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.746772    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.769772 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.769735    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-4805a7-serving-cert\""
Sep 18 19:05:49.811334 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.811296    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"]
Sep 18 19:05:49.836775 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.836734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aef3e227-586e-48a6-a3a0-275230efdbdc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.836969 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.836844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl6tw\" (UniqueName: \"kubernetes.io/projected/aef3e227-586e-48a6-a3a0-275230efdbdc-kube-api-access-sl6tw\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.836969 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.836892    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.836969 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.836923    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.938410 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.938368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aef3e227-586e-48a6-a3a0-275230efdbdc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.938632 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.938445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl6tw\" (UniqueName: \"kubernetes.io/projected/aef3e227-586e-48a6-a3a0-275230efdbdc-kube-api-access-sl6tw\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.938632 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.938478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.938632 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.938515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.938837 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:05:49.938677    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-serving-cert: secret "maas-api-e2e-shared-b-w2-4805a7-serving-cert" not found
Sep 18 19:05:49.938837 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:05:49.938770    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-maas-api-tls podName:aef3e227-586e-48a6-a3a0-275230efdbdc nodeName:}" failed. No retries permitted until 2026-09-18 19:05:50.438747953 +0000 UTC m=+1286.657755524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-maas-api-tls") pod "maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn" (UID: "aef3e227-586e-48a6-a3a0-275230efdbdc") : secret "maas-api-e2e-shared-b-w2-4805a7-serving-cert" not found
Sep 18 19:05:49.939211 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.939182    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aef3e227-586e-48a6-a3a0-275230efdbdc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.941455 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.941428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:49.956638 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:49.956595    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl6tw\" (UniqueName: \"kubernetes.io/projected/aef3e227-586e-48a6-a3a0-275230efdbdc-kube-api-access-sl6tw\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:50.443221 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:50.443182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:50.445910 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:50.445881    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:50.659635 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:50.659592    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:50.966596 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:50.966500    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"]
Sep 18 19:05:50.967575 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:05:50.967539    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef3e227_586e_48a6_a3a0_275230efdbdc.slice/crio-21da9245afa876b4c582f8b5d269731046104a934ca4bda24646b28466610edd WatchSource:0}: Error finding container 21da9245afa876b4c582f8b5d269731046104a934ca4bda24646b28466610edd: Status 404 returned error can't find the container with id 21da9245afa876b4c582f8b5d269731046104a934ca4bda24646b28466610edd
Sep 18 19:05:50.969296 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:50.969278    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:05:51.204943 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:51.204905    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn" event={"ID":"aef3e227-586e-48a6-a3a0-275230efdbdc","Type":"ContainerStarted","Data":"21da9245afa876b4c582f8b5d269731046104a934ca4bda24646b28466610edd"}
Sep 18 19:05:52.212731 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:52.212681    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn" event={"ID":"aef3e227-586e-48a6-a3a0-275230efdbdc","Type":"ContainerStarted","Data":"51df87864fa69f77b3784484a095280e675efc1a5b2d0ccb24a7f0f6447189aa"}
Sep 18 19:05:52.213140 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:52.212880    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:05:52.245565 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:52.245498    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn" podStartSLOduration=2.805641556 podStartE2EDuration="3.245476693s" totalImagesPullingTime="439.835137ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:05:52.242422447 +0000 UTC m=+1288.461430026" watchObservedRunningTime="2026-09-18 19:05:52.245476693 +0000 UTC m=+1288.464484273"
Sep 18 19:05:58.224538 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:05:58.224509    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:06:16.212927 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:16.212873    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 19:06:16.213448 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:16.212961    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 19:06:17.374366 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:17.374323    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerID="e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2" exitCode=0
Sep 18 19:06:17.374922 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:17.374400    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" event={"ID":"0d458cd2-a72f-4c7d-99a6-aa40698fd016","Type":"ContainerDied","Data":"e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2"}
Sep 18 19:06:17.374990 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:17.374948    2581 scope.go:117] "RemoveContainer" containerID="e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2"
Sep 18 19:06:17.376228 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:17.376197    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:06:17.376292 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:17.376280    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:06:17.376327 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:17.376316    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:06:18.383134 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:18.383098    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" event={"ID":"0d458cd2-a72f-4c7d-99a6-aa40698fd016","Type":"ContainerStarted","Data":"68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02"}
Sep 18 19:06:18.660410 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:18.660304    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:06:18.660595 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:18.660555    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 19:06:18.660652 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:18.660599    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 19:06:19.660946 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:19.660897    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 19:06:19.661493 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:19.660956    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 19:06:20.660331 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:20.660289    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 19:06:20.660565 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:20.660352    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 19:06:21.660243 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:21.660199    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 19:06:21.660662 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:21.660272    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 19:06:22.661003 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:22.660953    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 19:06:22.661426 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:22.661022    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 19:06:23.433247 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:23.433196    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7681-e78513"]
Sep 18 19:06:23.443075 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:23.443033    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-e78513"
Sep 18 19:06:23.446012 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:23.445983    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wgm9b\""
Sep 18 19:06:23.461069 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:23.461036    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7681-e78513"]
Sep 18 19:06:23.483918 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:23.483883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k6gc\" (UniqueName: \"kubernetes.io/projected/46247a07-9d81-48ca-ba61-089b01cfa2fc-kube-api-access-6k6gc\") pod \"test-curl-7681-e78513\" (UID: \"46247a07-9d81-48ca-ba61-089b01cfa2fc\") " pod="opendatahub/test-curl-7681-e78513"
Sep 18 19:06:23.585454 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:23.585419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k6gc\" (UniqueName: \"kubernetes.io/projected/46247a07-9d81-48ca-ba61-089b01cfa2fc-kube-api-access-6k6gc\") pod \"test-curl-7681-e78513\" (UID: \"46247a07-9d81-48ca-ba61-089b01cfa2fc\") " pod="opendatahub/test-curl-7681-e78513"
Sep 18 19:06:23.596283 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:23.596240    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k6gc\" (UniqueName: \"kubernetes.io/projected/46247a07-9d81-48ca-ba61-089b01cfa2fc-kube-api-access-6k6gc\") pod \"test-curl-7681-e78513\" (UID: \"46247a07-9d81-48ca-ba61-089b01cfa2fc\") " pod="opendatahub/test-curl-7681-e78513"
Sep 18 19:06:23.660491 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:23.660441    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 19:06:23.660680 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:23.660524    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 19:06:23.759251 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:23.759155    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-e78513"
Sep 18 19:06:24.080663 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:24.080620    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7681-e78513"]
Sep 18 19:06:24.080992 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:06:24.080967    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46247a07_9d81_48ca_ba61_089b01cfa2fc.slice/crio-32a74f627051979a242bec758034c0076ce92181d19056b69fc82fc1b908cd7d WatchSource:0}: Error finding container 32a74f627051979a242bec758034c0076ce92181d19056b69fc82fc1b908cd7d: Status 404 returned error can't find the container with id 32a74f627051979a242bec758034c0076ce92181d19056b69fc82fc1b908cd7d
Sep 18 19:06:24.422775 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:24.422736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7681-e78513" event={"ID":"46247a07-9d81-48ca-ba61-089b01cfa2fc","Type":"ContainerStarted","Data":"32a74f627051979a242bec758034c0076ce92181d19056b69fc82fc1b908cd7d"}
Sep 18 19:06:24.695080 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:24.694979    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:06:24.695533 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:24.695480    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:06:24.697499 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:24.697476    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:06:26.437213 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:26.437114    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7681-e78513" event={"ID":"46247a07-9d81-48ca-ba61-089b01cfa2fc","Type":"ContainerStarted","Data":"ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8"}
Sep 18 19:06:26.484950 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:26.484886    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7681-e78513" podStartSLOduration=1.397061839 podStartE2EDuration="3.484866428s" totalImagesPullingTime="2.087804589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:06:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:06:26.483343851 +0000 UTC m=+1322.702351429" watchObservedRunningTime="2026-09-18 19:06:26.484866428 +0000 UTC m=+1322.703874006"
Sep 18 19:06:27.010325 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.010288    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"]
Sep 18 19:06:27.010800 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.010767    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l" podUID="153e5cf1-aaaf-4f39-b2f1-bbe42379babe" containerName="maas-api" containerID="cri-o://8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee" gracePeriod=30
Sep 18 19:06:27.293026 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.292990    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 19:06:27.297323 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.297283    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7681-e78513"]
Sep 18 19:06:27.304476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.304398    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7681-e78513"]
Sep 18 19:06:27.324664 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.324627    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-openshift-service-ca\") pod \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") "
Sep 18 19:06:27.330357 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.324769    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-maas-api-tls\") pod \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") "
Sep 18 19:06:27.330357 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.324810    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-metrics-certs\") pod \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") "
Sep 18 19:06:27.330357 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.326016    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "153e5cf1-aaaf-4f39-b2f1-bbe42379babe" (UID: "153e5cf1-aaaf-4f39-b2f1-bbe42379babe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:27.330357 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.328051    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtq89\" (UniqueName: \"kubernetes.io/projected/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-kube-api-access-xtq89\") pod \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\" (UID: \"153e5cf1-aaaf-4f39-b2f1-bbe42379babe\") "
Sep 18 19:06:27.330357 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.328946    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:27.330357 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.329231    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "153e5cf1-aaaf-4f39-b2f1-bbe42379babe" (UID: "153e5cf1-aaaf-4f39-b2f1-bbe42379babe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:06:27.330357 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.329260    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "153e5cf1-aaaf-4f39-b2f1-bbe42379babe" (UID: "153e5cf1-aaaf-4f39-b2f1-bbe42379babe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:06:27.332096 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.332056    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-kube-api-access-xtq89" (OuterVolumeSpecName: "kube-api-access-xtq89") pod "153e5cf1-aaaf-4f39-b2f1-bbe42379babe" (UID: "153e5cf1-aaaf-4f39-b2f1-bbe42379babe"). InnerVolumeSpecName "kube-api-access-xtq89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:27.428445 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.428398    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7681-177b6f"]
Sep 18 19:06:27.429115 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.429092    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="153e5cf1-aaaf-4f39-b2f1-bbe42379babe" containerName="maas-api"
Sep 18 19:06:27.429182 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.429119    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="153e5cf1-aaaf-4f39-b2f1-bbe42379babe" containerName="maas-api"
Sep 18 19:06:27.429182 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.429148    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46247a07-9d81-48ca-ba61-089b01cfa2fc" containerName="test-curl-7681-e78513"
Sep 18 19:06:27.429182 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.429157    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="46247a07-9d81-48ca-ba61-089b01cfa2fc" containerName="test-curl-7681-e78513"
Sep 18 19:06:27.429316 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.429262    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="153e5cf1-aaaf-4f39-b2f1-bbe42379babe" containerName="maas-api"
Sep 18 19:06:27.429316 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.429277    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="46247a07-9d81-48ca-ba61-089b01cfa2fc" containerName="test-curl-7681-e78513"
Sep 18 19:06:27.430156 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.429980    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:27.430156 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.430012    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:27.430156 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.430028    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtq89\" (UniqueName: \"kubernetes.io/projected/153e5cf1-aaaf-4f39-b2f1-bbe42379babe-kube-api-access-xtq89\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:27.432955 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.432935    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-177b6f"
Sep 18 19:06:27.441658 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.441610    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7681-177b6f"]
Sep 18 19:06:27.446748 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.446678    2581 generic.go:356] "Generic (PLEG): container finished" podID="153e5cf1-aaaf-4f39-b2f1-bbe42379babe" containerID="8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee" exitCode=0
Sep 18 19:06:27.446748 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.446747    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"
Sep 18 19:06:27.447322 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.446741    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l" event={"ID":"153e5cf1-aaaf-4f39-b2f1-bbe42379babe","Type":"ContainerDied","Data":"8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee"}
Sep 18 19:06:27.447322 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:27.447069    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7681-e78513" container="test-curl-7681-e78513"
Sep 18 19:06:27.447322 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.447128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l" event={"ID":"153e5cf1-aaaf-4f39-b2f1-bbe42379babe","Type":"ContainerDied","Data":"fc05c9abc3ac995435bab5451c2443c1930eef5e54937ce7e494e0e7505836a5"}
Sep 18 19:06:27.447322 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.447167    2581 scope.go:117] "RemoveContainer" containerID="8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee"
Sep 18 19:06:27.460189 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.460162    2581 scope.go:117] "RemoveContainer" containerID="8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee"
Sep 18 19:06:27.460569 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:27.460548    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee\": container with ID starting with 8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee not found: ID does not exist" containerID="8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee"
Sep 18 19:06:27.460657 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.460576    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee"} err="failed to get container status \"8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee\": rpc error: code = NotFound desc = could not find container \"8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee\": container with ID starting with 8f2b8b7e3281c5954fc5d7f3402b2257b6ead2420b9abe88ca84ce6ef93ba0ee not found: ID does not exist"
Sep 18 19:06:27.486242 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.486209    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"]
Sep 18 19:06:27.504821 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.504783    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-15c3ed-5ff9cdd7fd-cxp9l"]
Sep 18 19:06:27.530868 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.530832    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwmbp\" (UniqueName: \"kubernetes.io/projected/b76cfd22-9236-4579-851a-d8e2a524dc14-kube-api-access-fwmbp\") pod \"test-curl-7681-177b6f\" (UID: \"b76cfd22-9236-4579-851a-d8e2a524dc14\") " pod="opendatahub/test-curl-7681-177b6f"
Sep 18 19:06:27.632188 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.632150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwmbp\" (UniqueName: \"kubernetes.io/projected/b76cfd22-9236-4579-851a-d8e2a524dc14-kube-api-access-fwmbp\") pod \"test-curl-7681-177b6f\" (UID: \"b76cfd22-9236-4579-851a-d8e2a524dc14\") " pod="opendatahub/test-curl-7681-177b6f"
Sep 18 19:06:27.646201 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.646162    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwmbp\" (UniqueName: \"kubernetes.io/projected/b76cfd22-9236-4579-851a-d8e2a524dc14-kube-api-access-fwmbp\") pod \"test-curl-7681-177b6f\" (UID: \"b76cfd22-9236-4579-851a-d8e2a524dc14\") " pod="opendatahub/test-curl-7681-177b6f"
Sep 18 19:06:27.745606 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:27.745501    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-177b6f"
Sep 18 19:06:28.034414 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:28.034384    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7681-177b6f"]
Sep 18 19:06:28.036696 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:06:28.036670    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb76cfd22_9236_4579_851a_d8e2a524dc14.slice/crio-45c254a1b8c1a4081cf9d3994c5e786e7e84e2762cd710b8feba6f962dccd19e WatchSource:0}: Error finding container 45c254a1b8c1a4081cf9d3994c5e786e7e84e2762cd710b8feba6f962dccd19e: Status 404 returned error can't find the container with id 45c254a1b8c1a4081cf9d3994c5e786e7e84e2762cd710b8feba6f962dccd19e
Sep 18 19:06:28.315090 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:28.315002    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="153e5cf1-aaaf-4f39-b2f1-bbe42379babe" path="/var/lib/kubelet/pods/153e5cf1-aaaf-4f39-b2f1-bbe42379babe/volumes"
Sep 18 19:06:28.455740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:28.455682    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7681-177b6f" event={"ID":"b76cfd22-9236-4579-851a-d8e2a524dc14","Type":"ContainerStarted","Data":"939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e"}
Sep 18 19:06:28.455740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:28.455746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7681-177b6f" event={"ID":"b76cfd22-9236-4579-851a-d8e2a524dc14","Type":"ContainerStarted","Data":"45c254a1b8c1a4081cf9d3994c5e786e7e84e2762cd710b8feba6f962dccd19e"}
Sep 18 19:06:28.456241 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:28.455775    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7681-e78513" podUID="46247a07-9d81-48ca-ba61-089b01cfa2fc" containerName="test-curl-7681-e78513" containerID="cri-o://ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8" gracePeriod=2
Sep 18 19:06:28.492149 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:28.492087    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7681-177b6f" podStartSLOduration=1.492068361 podStartE2EDuration="1.492068361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:06:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:06:28.489383055 +0000 UTC m=+1324.708390634" watchObservedRunningTime="2026-09-18 19:06:28.492068361 +0000 UTC m=+1324.711075938"
Sep 18 19:06:29.380729 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.380602    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7681-177b6f"]
Sep 18 19:06:29.385955 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.385919    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7681-177b6f"]
Sep 18 19:06:29.461391 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:29.461362    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7681-177b6f" container="test-curl-7681-177b6f"
Sep 18 19:06:29.534005 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.533963    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7681-936610"]
Sep 18 19:06:29.534422 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.534406    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b76cfd22-9236-4579-851a-d8e2a524dc14" containerName="test-curl-7681-177b6f"
Sep 18 19:06:29.534481 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.534423    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b76cfd22-9236-4579-851a-d8e2a524dc14" containerName="test-curl-7681-177b6f"
Sep 18 19:06:29.534517 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.534503    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b76cfd22-9236-4579-851a-d8e2a524dc14" containerName="test-curl-7681-177b6f"
Sep 18 19:06:29.538162 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.538132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-936610"
Sep 18 19:06:29.547919 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.547893    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7681-936610"]
Sep 18 19:06:29.650972 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.650877    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5dth\" (UniqueName: \"kubernetes.io/projected/a00343b5-e0cd-4340-b35e-c0feadee113f-kube-api-access-x5dth\") pod \"test-curl-7681-936610\" (UID: \"a00343b5-e0cd-4340-b35e-c0feadee113f\") " pod="opendatahub/test-curl-7681-936610"
Sep 18 19:06:29.751820 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.751774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5dth\" (UniqueName: \"kubernetes.io/projected/a00343b5-e0cd-4340-b35e-c0feadee113f-kube-api-access-x5dth\") pod \"test-curl-7681-936610\" (UID: \"a00343b5-e0cd-4340-b35e-c0feadee113f\") " pod="opendatahub/test-curl-7681-936610"
Sep 18 19:06:29.780785 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.780744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5dth\" (UniqueName: \"kubernetes.io/projected/a00343b5-e0cd-4340-b35e-c0feadee113f-kube-api-access-x5dth\") pod \"test-curl-7681-936610\" (UID: \"a00343b5-e0cd-4340-b35e-c0feadee113f\") " pod="opendatahub/test-curl-7681-936610"
Sep 18 19:06:29.850442 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:29.850396    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-936610"
Sep 18 19:06:30.138047 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:30.138020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7681-936610"]
Sep 18 19:06:30.140331 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:06:30.140287    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda00343b5_e0cd_4340_b35e_c0feadee113f.slice/crio-42469ef99010678e5c8717b26bd6f808ea05fc655dc0158ae115e89a3083c474 WatchSource:0}: Error finding container 42469ef99010678e5c8717b26bd6f808ea05fc655dc0158ae115e89a3083c474: Status 404 returned error can't find the container with id 42469ef99010678e5c8717b26bd6f808ea05fc655dc0158ae115e89a3083c474
Sep 18 19:06:30.471394 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:30.471357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7681-936610" event={"ID":"a00343b5-e0cd-4340-b35e-c0feadee113f","Type":"ContainerStarted","Data":"0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643"}
Sep 18 19:06:30.471890 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:30.471409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7681-936610" event={"ID":"a00343b5-e0cd-4340-b35e-c0feadee113f","Type":"ContainerStarted","Data":"42469ef99010678e5c8717b26bd6f808ea05fc655dc0158ae115e89a3083c474"}
Sep 18 19:06:30.471890 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:30.471436    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7681-177b6f" podUID="b76cfd22-9236-4579-851a-d8e2a524dc14" containerName="test-curl-7681-177b6f" containerID="cri-o://939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e" gracePeriod=2
Sep 18 19:06:30.506263 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:30.506206    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7681-936610" podStartSLOduration=1.506186724 podStartE2EDuration="1.506186724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:06:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:06:30.505620603 +0000 UTC m=+1326.724628186" watchObservedRunningTime="2026-09-18 19:06:30.506186724 +0000 UTC m=+1326.725194301"
Sep 18 19:06:30.602755 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:30.602696    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-e78513"
Sep 18 19:06:30.660538 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:30.660503    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k6gc\" (UniqueName: \"kubernetes.io/projected/46247a07-9d81-48ca-ba61-089b01cfa2fc-kube-api-access-6k6gc\") pod \"46247a07-9d81-48ca-ba61-089b01cfa2fc\" (UID: \"46247a07-9d81-48ca-ba61-089b01cfa2fc\") "
Sep 18 19:06:30.662996 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:30.662965    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46247a07-9d81-48ca-ba61-089b01cfa2fc-kube-api-access-6k6gc" (OuterVolumeSpecName: "kube-api-access-6k6gc") pod "46247a07-9d81-48ca-ba61-089b01cfa2fc" (UID: "46247a07-9d81-48ca-ba61-089b01cfa2fc"). InnerVolumeSpecName "kube-api-access-6k6gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:30.762031 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:30.761928    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k6gc\" (UniqueName: \"kubernetes.io/projected/46247a07-9d81-48ca-ba61-089b01cfa2fc-kube-api-access-6k6gc\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:31.346429 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:31.346334    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7681-936610"]
Sep 18 19:06:31.349680 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:31.349644    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7681-936610"]
Sep 18 19:06:31.479717 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:31.479664    2581 generic.go:356] "Generic (PLEG): container finished" podID="46247a07-9d81-48ca-ba61-089b01cfa2fc" containerID="ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8" exitCode=137
Sep 18 19:06:31.480180 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:31.479746    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-e78513"
Sep 18 19:06:31.480180 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:31.479773    2581 scope.go:117] "RemoveContainer" containerID="ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8"
Sep 18 19:06:31.480180 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:31.480142    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7681-936610" container="test-curl-7681-936610"
Sep 18 19:06:31.490160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:31.490135    2581 scope.go:117] "RemoveContainer" containerID="ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8"
Sep 18 19:06:31.490569 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:31.490546    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8\": container with ID starting with ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8 not found: ID does not exist" containerID="ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8"
Sep 18 19:06:31.490649 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:31.490580    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8"} err="failed to get container status \"ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8\": rpc error: code = NotFound desc = could not find container \"ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8\": container with ID starting with ad395c24c65c4e0c6a40f8629a41047d5b8562d113db86ebdc396a00943da7c8 not found: ID does not exist"
Sep 18 19:06:31.490831 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:31.490809    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7681-e78513\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="46247a07-9d81-48ca-ba61-089b01cfa2fc" pod="opendatahub/test-curl-7681-e78513"
Sep 18 19:06:32.314717 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.314656    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46247a07-9d81-48ca-ba61-089b01cfa2fc" path="/var/lib/kubelet/pods/46247a07-9d81-48ca-ba61-089b01cfa2fc/volumes"
Sep 18 19:06:32.483754 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.483718    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"]
Sep 18 19:06:32.484208 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.484145    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a00343b5-e0cd-4340-b35e-c0feadee113f" containerName="test-curl-7681-936610"
Sep 18 19:06:32.484208 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.484156    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a00343b5-e0cd-4340-b35e-c0feadee113f" containerName="test-curl-7681-936610"
Sep 18 19:06:32.484323 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.484230    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a00343b5-e0cd-4340-b35e-c0feadee113f" containerName="test-curl-7681-936610"
Sep 18 19:06:32.489320 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.489291    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.491693 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.491653    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7681-936610" podUID="a00343b5-e0cd-4340-b35e-c0feadee113f" containerName="test-curl-7681-936610" containerID="cri-o://0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643" gracePeriod=2
Sep 18 19:06:32.493575 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.493550    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4ac9ac7d-openshift-default-dockercfg-zzml6\""
Sep 18 19:06:32.518650 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.518615    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"]
Sep 18 19:06:32.581831 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.581656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-token\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.581831 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.581802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-certs\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.582160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.581885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d87rm\" (UniqueName: \"kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-kube-api-access-d87rm\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.582160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.581960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istiod-ca-cert\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.582160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.581990    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-envoy\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.582160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.582073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-data\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.582160 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.582107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-credential-socket\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.582432 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.582235    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-socket\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.582432 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.582324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-podinfo\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.682948 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.682909    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-socket\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683138 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.682971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-podinfo\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683138 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-token\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683138 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-certs\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683138 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d87rm\" (UniqueName: \"kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-kube-api-access-d87rm\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683138 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istiod-ca-cert\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683403 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-envoy\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683403 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-data\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683403 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683235    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-credential-socket\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683403 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-socket\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683617 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683404    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-certs\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683674 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-credential-socket\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.683819 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.683794    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-data\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.684293 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.684269    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istiod-ca-cert\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.685689 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.685670    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-envoy\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.686002 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.685983    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-podinfo\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.687965 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.687943    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-177b6f"
Sep 18 19:06:32.715040 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.715012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-token\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.731187 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.731140    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d87rm\" (UniqueName: \"kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-kube-api-access-d87rm\") pod \"e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") " pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.784193 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.784157    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwmbp\" (UniqueName: \"kubernetes.io/projected/b76cfd22-9236-4579-851a-d8e2a524dc14-kube-api-access-fwmbp\") pod \"b76cfd22-9236-4579-851a-d8e2a524dc14\" (UID: \"b76cfd22-9236-4579-851a-d8e2a524dc14\") "
Sep 18 19:06:32.786485 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.786452    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76cfd22-9236-4579-851a-d8e2a524dc14-kube-api-access-fwmbp" (OuterVolumeSpecName: "kube-api-access-fwmbp") pod "b76cfd22-9236-4579-851a-d8e2a524dc14" (UID: "b76cfd22-9236-4579-851a-d8e2a524dc14"). InnerVolumeSpecName "kube-api-access-fwmbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:32.849103 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.849002    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:32.885384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:32.885345    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwmbp\" (UniqueName: \"kubernetes.io/projected/b76cfd22-9236-4579-851a-d8e2a524dc14-kube-api-access-fwmbp\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:33.154941 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:06:33.154911    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44613b45_0523_4bb5_ac8e_0fec3d7cb093.slice/crio-34e34e01ea4836178a239a037dd35c692f861fe508513eb15b86968d1e6c634b WatchSource:0}: Error finding container 34e34e01ea4836178a239a037dd35c692f861fe508513eb15b86968d1e6c634b: Status 404 returned error can't find the container with id 34e34e01ea4836178a239a037dd35c692f861fe508513eb15b86968d1e6c634b
Sep 18 19:06:33.156324 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.156298    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"]
Sep 18 19:06:33.158602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.158548    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:06:33.158755 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.158695    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:06:33.164425 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.158825    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:06:33.500333 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.500279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" event={"ID":"44613b45-0523-4bb5-ac8e-0fec3d7cb093","Type":"ContainerStarted","Data":"0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9"}
Sep 18 19:06:33.500870 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.500833    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" event={"ID":"44613b45-0523-4bb5-ac8e-0fec3d7cb093","Type":"ContainerStarted","Data":"34e34e01ea4836178a239a037dd35c692f861fe508513eb15b86968d1e6c634b"}
Sep 18 19:06:33.502188 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.502159    2581 generic.go:356] "Generic (PLEG): container finished" podID="b76cfd22-9236-4579-851a-d8e2a524dc14" containerID="939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e" exitCode=137
Sep 18 19:06:33.502342 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.502219    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-177b6f"
Sep 18 19:06:33.502342 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.502255    2581 scope.go:117] "RemoveContainer" containerID="939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e"
Sep 18 19:06:33.514565 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.514539    2581 scope.go:117] "RemoveContainer" containerID="939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e"
Sep 18 19:06:33.515023 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:33.515001    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e\": container with ID starting with 939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e not found: ID does not exist" containerID="939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e"
Sep 18 19:06:33.515096 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.515036    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e"} err="failed to get container status \"939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e\": rpc error: code = NotFound desc = could not find container \"939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e\": container with ID starting with 939624f6cb715669c31ff6665fba396d9d34044d4070b7ddfae7e843f365ba5e not found: ID does not exist"
Sep 18 19:06:33.553080 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.553017    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" podStartSLOduration=1.5529993530000001 podStartE2EDuration="1.552999353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:06:33.552338118 +0000 UTC m=+1329.771345706" watchObservedRunningTime="2026-09-18 19:06:33.552999353 +0000 UTC m=+1329.772006991"
Sep 18 19:06:33.555524 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:33.555489    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7681-177b6f\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="b76cfd22-9236-4579-851a-d8e2a524dc14" pod="opendatahub/test-curl-7681-177b6f"
Sep 18 19:06:33.849819 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:33.849719    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:34.315332 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:34.315297    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b76cfd22-9236-4579-851a-d8e2a524dc14" path="/var/lib/kubelet/pods/b76cfd22-9236-4579-851a-d8e2a524dc14/volumes"
Sep 18 19:06:34.316138 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:34.316109    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7681-177b6f\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="b76cfd22-9236-4579-851a-d8e2a524dc14" pod="opendatahub/test-curl-7681-177b6f"
Sep 18 19:06:34.698384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:34.698349    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-936610"
Sep 18 19:06:34.804670 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:34.804620    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5dth\" (UniqueName: \"kubernetes.io/projected/a00343b5-e0cd-4340-b35e-c0feadee113f-kube-api-access-x5dth\") pod \"a00343b5-e0cd-4340-b35e-c0feadee113f\" (UID: \"a00343b5-e0cd-4340-b35e-c0feadee113f\") "
Sep 18 19:06:34.806993 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:34.806959    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a00343b5-e0cd-4340-b35e-c0feadee113f-kube-api-access-x5dth" (OuterVolumeSpecName: "kube-api-access-x5dth") pod "a00343b5-e0cd-4340-b35e-c0feadee113f" (UID: "a00343b5-e0cd-4340-b35e-c0feadee113f"). InnerVolumeSpecName "kube-api-access-x5dth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:34.855111 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:34.855075    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:34.906388 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:34.906279    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5dth\" (UniqueName: \"kubernetes.io/projected/a00343b5-e0cd-4340-b35e-c0feadee113f-kube-api-access-x5dth\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:35.517525 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:35.517490    2581 generic.go:356] "Generic (PLEG): container finished" podID="a00343b5-e0cd-4340-b35e-c0feadee113f" containerID="0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643" exitCode=137
Sep 18 19:06:35.517732 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:35.517543    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7681-936610"
Sep 18 19:06:35.517732 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:35.517602    2581 scope.go:117] "RemoveContainer" containerID="0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643"
Sep 18 19:06:35.518047 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:35.517983    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:35.519275 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:35.519248    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:06:35.530171 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:35.529954    2581 scope.go:117] "RemoveContainer" containerID="0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643"
Sep 18 19:06:35.530394 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:35.530368    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643\": container with ID starting with 0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643 not found: ID does not exist" containerID="0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643"
Sep 18 19:06:35.530466 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:35.530404    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643"} err="failed to get container status \"0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643\": rpc error: code = NotFound desc = could not find container \"0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643\": container with ID starting with 0ec76fed4cbd196ee76286752a2459ffddf170d59dff356d193c876362db8643 not found: ID does not exist"
Sep 18 19:06:35.549846 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:35.549798    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7681-936610\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="a00343b5-e0cd-4340-b35e-c0feadee113f" pod="opendatahub/test-curl-7681-936610"
Sep 18 19:06:36.318056 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:36.318015    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a00343b5-e0cd-4340-b35e-c0feadee113f" path="/var/lib/kubelet/pods/a00343b5-e0cd-4340-b35e-c0feadee113f/volumes"
Sep 18 19:06:42.498342 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.498301    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"]
Sep 18 19:06:42.503471 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.503444    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.511825 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.511802    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1a444c\"/\"distinct-2-w0-1a444c-kserve-self-signed-certs\""
Sep 18 19:06:42.525793 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.525742    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"]
Sep 18 19:06:42.589329 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.589285    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-home\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.589329 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.589349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tmp-dir\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.589590 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.589460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8rcp\" (UniqueName: \"kubernetes.io/projected/64a91386-f39a-4807-a16a-bf86ccd8ec0e-kube-api-access-q8rcp\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.589590 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.589515    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-model-cache\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.589590 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.589578    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tls-certs\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.589752 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.589623    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-dshm\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.690620 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.690584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-home\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.690831 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.690652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tmp-dir\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.690831 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.690779    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8rcp\" (UniqueName: \"kubernetes.io/projected/64a91386-f39a-4807-a16a-bf86ccd8ec0e-kube-api-access-q8rcp\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.690956 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.690828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-model-cache\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.690956 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.690870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tls-certs\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.690956 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.690922    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-dshm\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.691107 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.691093    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-home\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.691186 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.691112    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tmp-dir\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.691297 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.691226    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-model-cache\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.693341 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.693317    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-dshm\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.693814 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.693786    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tls-certs\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.716674 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.716633    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"]
Sep 18 19:06:42.719126 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.719059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8rcp\" (UniqueName: \"kubernetes.io/projected/64a91386-f39a-4807-a16a-bf86ccd8ec0e-kube-api-access-q8rcp\") pod \"distinct-2-w0-1a444c-kserve-868767d5c5-qx22r\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") " pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.721508 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.721477    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.725252 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.725019    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4ac9ac7d-serving-cert\""
Sep 18 19:06:42.737139 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.737097    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"]
Sep 18 19:06:42.813495 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.813397    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:42.893805 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.893761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-metrics-certs\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.894021 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.893825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-maas-api-tls\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.894021 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.893869    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww82v\" (UniqueName: \"kubernetes.io/projected/881a3d78-9d3b-430d-b30d-935a73344ff0-kube-api-access-ww82v\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.894021 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.893921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/881a3d78-9d3b-430d-b30d-935a73344ff0-openshift-service-ca\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.994697 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.994649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-metrics-certs\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.994697 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.994718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-maas-api-tls\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.994994 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.994748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww82v\" (UniqueName: \"kubernetes.io/projected/881a3d78-9d3b-430d-b30d-935a73344ff0-kube-api-access-ww82v\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.994994 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.994783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/881a3d78-9d3b-430d-b30d-935a73344ff0-openshift-service-ca\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.995578 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.995497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/881a3d78-9d3b-430d-b30d-935a73344ff0-openshift-service-ca\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.997815 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.997788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-metrics-certs\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:42.997815 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:42.997803    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-maas-api-tls\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:43.005189 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:43.005149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww82v\" (UniqueName: \"kubernetes.io/projected/881a3d78-9d3b-430d-b30d-935a73344ff0-kube-api-access-ww82v\") pod \"maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:43.046032 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:43.045886    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:43.117540 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:43.117235    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"]
Sep 18 19:06:43.120791 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:06:43.120755    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64a91386_f39a_4807_a16a_bf86ccd8ec0e.slice/crio-9227d6f9f7cb27759351a211dfcd19f89446e52d7f419745a75dffee5d32a3fa WatchSource:0}: Error finding container 9227d6f9f7cb27759351a211dfcd19f89446e52d7f419745a75dffee5d32a3fa: Status 404 returned error can't find the container with id 9227d6f9f7cb27759351a211dfcd19f89446e52d7f419745a75dffee5d32a3fa
Sep 18 19:06:43.343964 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:43.343926    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"]
Sep 18 19:06:43.345019 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:06:43.344987    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881a3d78_9d3b_430d_b30d_935a73344ff0.slice/crio-6b09ec785eafdba4ffaf5ae073cbda196baf66c68f24ff1ab0836b289d7894d3 WatchSource:0}: Error finding container 6b09ec785eafdba4ffaf5ae073cbda196baf66c68f24ff1ab0836b289d7894d3: Status 404 returned error can't find the container with id 6b09ec785eafdba4ffaf5ae073cbda196baf66c68f24ff1ab0836b289d7894d3
Sep 18 19:06:43.575899 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:43.575850    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d" event={"ID":"881a3d78-9d3b-430d-b30d-935a73344ff0","Type":"ContainerStarted","Data":"6b09ec785eafdba4ffaf5ae073cbda196baf66c68f24ff1ab0836b289d7894d3"}
Sep 18 19:06:43.577920 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:43.577858    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r" event={"ID":"64a91386-f39a-4807-a16a-bf86ccd8ec0e","Type":"ContainerStarted","Data":"96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d"}
Sep 18 19:06:43.578083 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:43.577934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r" event={"ID":"64a91386-f39a-4807-a16a-bf86ccd8ec0e","Type":"ContainerStarted","Data":"9227d6f9f7cb27759351a211dfcd19f89446e52d7f419745a75dffee5d32a3fa"}
Sep 18 19:06:43.621489 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:43.621419    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r" podStartSLOduration=1.621400876 podStartE2EDuration="1.621400876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:06:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:06:43.618754105 +0000 UTC m=+1339.837761688" watchObservedRunningTime="2026-09-18 19:06:43.621400876 +0000 UTC m=+1339.840408453"
Sep 18 19:06:44.586383 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:44.586340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d" event={"ID":"881a3d78-9d3b-430d-b30d-935a73344ff0","Type":"ContainerStarted","Data":"e2ac3779006edecfa36ba541223fe27ac412dbc1e0bb1c6748f2db221ea9fb28"}
Sep 18 19:06:44.586919 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:44.586520    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:44.646033 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:44.645939    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d" podStartSLOduration=2.240286568 podStartE2EDuration="2.645921442s" totalImagesPullingTime="405.634874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:06:44.645688472 +0000 UTC m=+1340.864696063" watchObservedRunningTime="2026-09-18 19:06:44.645921442 +0000 UTC m=+1340.864929020"
Sep 18 19:06:47.390213 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:47.390170    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"]
Sep 18 19:06:49.647561 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:49.647479    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"]
Sep 18 19:06:49.648105 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:49.647730    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q" podUID="11f9884b-c729-47bd-aa5e-3e1ad77a18f2" containerName="maas-api" containerID="cri-o://78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b" gracePeriod=30
Sep 18 19:06:49.679288 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:49.679236    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q" podUID="11f9884b-c729-47bd-aa5e-3e1ad77a18f2" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.52:8443/health\": dial tcp 10.132.0.52:8443: connect: connection refused"
Sep 18 19:06:49.909955 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:49.909841    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 19:06:50.074565 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.074529    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-maas-api-tls\") pod \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") "
Sep 18 19:06:50.074772 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.074640    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-openshift-service-ca\") pod \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") "
Sep 18 19:06:50.074772 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.074726    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-metrics-certs\") pod \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") "
Sep 18 19:06:50.074772 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.074759    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4hdw\" (UniqueName: \"kubernetes.io/projected/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-kube-api-access-t4hdw\") pod \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\" (UID: \"11f9884b-c729-47bd-aa5e-3e1ad77a18f2\") "
Sep 18 19:06:50.075274 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.075227    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11f9884b-c729-47bd-aa5e-3e1ad77a18f2" (UID: "11f9884b-c729-47bd-aa5e-3e1ad77a18f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:50.077579 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.077378    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11f9884b-c729-47bd-aa5e-3e1ad77a18f2" (UID: "11f9884b-c729-47bd-aa5e-3e1ad77a18f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:06:50.077818 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.077594    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "11f9884b-c729-47bd-aa5e-3e1ad77a18f2" (UID: "11f9884b-c729-47bd-aa5e-3e1ad77a18f2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:06:50.077818 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.077675    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-kube-api-access-t4hdw" (OuterVolumeSpecName: "kube-api-access-t4hdw") pod "11f9884b-c729-47bd-aa5e-3e1ad77a18f2" (UID: "11f9884b-c729-47bd-aa5e-3e1ad77a18f2"). InnerVolumeSpecName "kube-api-access-t4hdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:50.176740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.176662    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:50.176740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.176742    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4hdw\" (UniqueName: \"kubernetes.io/projected/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-kube-api-access-t4hdw\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:50.176972 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.176757    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:50.176972 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.176767    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11f9884b-c729-47bd-aa5e-3e1ad77a18f2-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:06:50.598188 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.598109    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:06:50.626684 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.626651    2581 generic.go:356] "Generic (PLEG): container finished" podID="11f9884b-c729-47bd-aa5e-3e1ad77a18f2" containerID="78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b" exitCode=0
Sep 18 19:06:50.626897 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.626737    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"
Sep 18 19:06:50.626897 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.626739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q" event={"ID":"11f9884b-c729-47bd-aa5e-3e1ad77a18f2","Type":"ContainerDied","Data":"78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b"}
Sep 18 19:06:50.626897 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.626894    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q" event={"ID":"11f9884b-c729-47bd-aa5e-3e1ad77a18f2","Type":"ContainerDied","Data":"54e064d0f835699c80184a66e5f27821554ed92dd9ebf15d89a49ffa22055e48"}
Sep 18 19:06:50.627022 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.626921    2581 scope.go:117] "RemoveContainer" containerID="78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b"
Sep 18 19:06:50.638121 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.638101    2581 scope.go:117] "RemoveContainer" containerID="78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b"
Sep 18 19:06:50.638529 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:06:50.638506    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b\": container with ID starting with 78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b not found: ID does not exist" containerID="78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b"
Sep 18 19:06:50.638625 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.638546    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b"} err="failed to get container status \"78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b\": rpc error: code = NotFound desc = could not find container \"78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b\": container with ID starting with 78d320398cf9c09952c6f53e3e24ab18d79b7b2b4c10b3ab0787114828b9501b not found: ID does not exist"
Sep 18 19:06:50.701801 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.701765    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"]
Sep 18 19:06:50.751138 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:50.751078    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a81e5-8c87d85cf-54j8q"]
Sep 18 19:06:52.314029 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:52.313995    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11f9884b-c729-47bd-aa5e-3e1ad77a18f2" path="/var/lib/kubelet/pods/11f9884b-c729-47bd-aa5e-3e1ad77a18f2/volumes"
Sep 18 19:06:52.814125 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:52.814069    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:52.814125 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:52.814131    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:52.828319 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:52.828287    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:06:53.658957 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:06:53.658925    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:07:02.410895 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.410830    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7" podUID="e6582d3e-2c2a-4c3d-acab-d0781368bad4" containerName="main" containerID="cri-o://f8dd8c6234403904281638a2c41572c297711f02f9af4dca65b93c5ac9a928eb" gracePeriod=45
Sep 18 19:07:02.708433 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.708397    2581 generic.go:356] "Generic (PLEG): container finished" podID="e6582d3e-2c2a-4c3d-acab-d0781368bad4" containerID="f8dd8c6234403904281638a2c41572c297711f02f9af4dca65b93c5ac9a928eb" exitCode=0
Sep 18 19:07:02.708607 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.708474    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7" event={"ID":"e6582d3e-2c2a-4c3d-acab-d0781368bad4","Type":"ContainerDied","Data":"f8dd8c6234403904281638a2c41572c297711f02f9af4dca65b93c5ac9a928eb"}
Sep 18 19:07:02.708607 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.708521    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7" event={"ID":"e6582d3e-2c2a-4c3d-acab-d0781368bad4","Type":"ContainerDied","Data":"cbcf110c774a8a201e90bb185bf75cadcc837a73ba284576a9620ab706ba4e8c"}
Sep 18 19:07:02.708607 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.708532    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbcf110c774a8a201e90bb185bf75cadcc837a73ba284576a9620ab706ba4e8c"
Sep 18 19:07:02.720940 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.720904    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:07:02.810234 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.810190    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-dshm\") pod \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") "
Sep 18 19:07:02.810234 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.810242    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxjwp\" (UniqueName: \"kubernetes.io/projected/e6582d3e-2c2a-4c3d-acab-d0781368bad4-kube-api-access-sxjwp\") pod \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") "
Sep 18 19:07:02.810515 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.810305    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tmp-dir\") pod \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") "
Sep 18 19:07:02.810515 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.810324    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-model-cache\") pod \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") "
Sep 18 19:07:02.810515 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.810342    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-home\") pod \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") "
Sep 18 19:07:02.810515 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.810372    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tls-certs\") pod \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\" (UID: \"e6582d3e-2c2a-4c3d-acab-d0781368bad4\") "
Sep 18 19:07:02.810753 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.810539    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6582d3e-2c2a-4c3d-acab-d0781368bad4" (UID: "e6582d3e-2c2a-4c3d-acab-d0781368bad4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:02.810753 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.810553    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6582d3e-2c2a-4c3d-acab-d0781368bad4" (UID: "e6582d3e-2c2a-4c3d-acab-d0781368bad4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:02.810753 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.810627    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-home" (OuterVolumeSpecName: "home") pod "e6582d3e-2c2a-4c3d-acab-d0781368bad4" (UID: "e6582d3e-2c2a-4c3d-acab-d0781368bad4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:02.813157 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.813118    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-dshm" (OuterVolumeSpecName: "dshm") pod "e6582d3e-2c2a-4c3d-acab-d0781368bad4" (UID: "e6582d3e-2c2a-4c3d-acab-d0781368bad4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:02.813386 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.813168    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6582d3e-2c2a-4c3d-acab-d0781368bad4" (UID: "e6582d3e-2c2a-4c3d-acab-d0781368bad4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:07:02.813566 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.813543    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6582d3e-2c2a-4c3d-acab-d0781368bad4-kube-api-access-sxjwp" (OuterVolumeSpecName: "kube-api-access-sxjwp") pod "e6582d3e-2c2a-4c3d-acab-d0781368bad4" (UID: "e6582d3e-2c2a-4c3d-acab-d0781368bad4"). InnerVolumeSpecName "kube-api-access-sxjwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:07:02.912017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.911973    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxjwp\" (UniqueName: \"kubernetes.io/projected/e6582d3e-2c2a-4c3d-acab-d0781368bad4-kube-api-access-sxjwp\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:07:02.912017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.912010    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:07:02.912017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.912020    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:07:02.912017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.912029    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:07:02.912299 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.912037    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6582d3e-2c2a-4c3d-acab-d0781368bad4-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:07:02.912299 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:02.912044    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6582d3e-2c2a-4c3d-acab-d0781368bad4-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:07:03.716133 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:03.716097    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"
Sep 18 19:07:03.787355 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:03.787312    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"]
Sep 18 19:07:03.799846 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:03.799801    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-3a81e5/test-model-3a81e5-kserve-86c45b659-2ntf7"]
Sep 18 19:07:04.321427 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:04.321387    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6582d3e-2c2a-4c3d-acab-d0781368bad4" path="/var/lib/kubelet/pods/e6582d3e-2c2a-4c3d-acab-d0781368bad4/volumes"
Sep 18 19:07:06.560924 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:06.560870    2581 patch_prober.go:28] interesting pod/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 19:07:06.561344 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:06.560963    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 19:07:07.748449 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:07.748410    2581 generic.go:356] "Generic (PLEG): container finished" podID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerID="0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9" exitCode=0
Sep 18 19:07:07.748903 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:07.748478    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" event={"ID":"44613b45-0523-4bb5-ac8e-0fec3d7cb093","Type":"ContainerDied","Data":"0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9"}
Sep 18 19:07:07.748971 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:07.748953    2581 scope.go:117] "RemoveContainer" containerID="0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9"
Sep 18 19:07:07.750215 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:07.750185    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:07:07.750287 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:07.750269    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:07:07.750326 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:07.750315    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:07:08.759844 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:08.759806    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" event={"ID":"44613b45-0523-4bb5-ac8e-0fec3d7cb093","Type":"ContainerStarted","Data":"02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a"}
Sep 18 19:07:08.850362 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:08.850314    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:07:08.850586 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:08.850458    2581 patch_prober.go:28] interesting pod/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:07:08.850586 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:08.850503    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 19:07:09.849918 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:09.849878    2581 patch_prober.go:28] interesting pod/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:07:09.850548 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:09.850513    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 19:07:10.849605 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:10.849553    2581 patch_prober.go:28] interesting pod/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:07:10.849876 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:10.849640    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 19:07:11.850069 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:11.850022    2581 patch_prober.go:28] interesting pod/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:07:11.850597 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:11.850104    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 19:07:12.850509 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:12.850455    2581 patch_prober.go:28] interesting pod/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:07:12.851054 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:12.850534    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 19:07:13.850388 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:13.850289    2581 patch_prober.go:28] interesting pod/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:07:13.850388 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:13.850368    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 19:07:14.854382 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:14.854347    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:07:14.854822 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:14.854649    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:07:14.855670 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:07:14.855646    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:08:09.181589 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.181548    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"]
Sep 18 19:08:09.182313 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.182019    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6582d3e-2c2a-4c3d-acab-d0781368bad4" containerName="main"
Sep 18 19:08:09.182313 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.182033    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6582d3e-2c2a-4c3d-acab-d0781368bad4" containerName="main"
Sep 18 19:08:09.182313 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.182045    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11f9884b-c729-47bd-aa5e-3e1ad77a18f2" containerName="maas-api"
Sep 18 19:08:09.182313 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.182051    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="11f9884b-c729-47bd-aa5e-3e1ad77a18f2" containerName="maas-api"
Sep 18 19:08:09.182313 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.182148    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6582d3e-2c2a-4c3d-acab-d0781368bad4" containerName="main"
Sep 18 19:08:09.182313 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.182161    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="11f9884b-c729-47bd-aa5e-3e1ad77a18f2" containerName="maas-api"
Sep 18 19:08:09.185020 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.184999    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.189875 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.189852    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-19944d-openshift-default-dockercfg-xfxz8\""
Sep 18 19:08:09.200650 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.200616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-credential-socket\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.200650 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.200651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-certs\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.200860 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.200676    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-socket\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.200860 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.200830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-podinfo\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.200954 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.200860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-token\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.200954 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.200880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afd77893-0ff8-4b2e-95fa-7dca11d72330-istiod-ca-cert\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.201020 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.200955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnwxj\" (UniqueName: \"kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-kube-api-access-dnwxj\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.201020 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.200990    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-envoy\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.201087 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.201036    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-data\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.215060 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.215022    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"]
Sep 18 19:08:09.301840 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.301803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-podinfo\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302063 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.301848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-token\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302063 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.301870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afd77893-0ff8-4b2e-95fa-7dca11d72330-istiod-ca-cert\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302063 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.301929    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnwxj\" (UniqueName: \"kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-kube-api-access-dnwxj\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302063 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.301986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-envoy\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302063 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.302036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-data\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302360 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.302112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-credential-socket\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302360 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.302140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-certs\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302360 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.302184    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-socket\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302723 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.302611    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-credential-socket\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302723 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.302633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-certs\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302904 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.302764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-data\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.302904 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.302864    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-socket\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.303419 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.303309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afd77893-0ff8-4b2e-95fa-7dca11d72330-istiod-ca-cert\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.305077 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.305055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-envoy\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.305532 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.305490    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-podinfo\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.318088 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.318059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-token\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.318483 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.318462    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnwxj\" (UniqueName: \"kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-kube-api-access-dnwxj\") pod \"e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") " pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.495867 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.495771    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:09.814833 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.814801    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"]
Sep 18 19:08:09.816201 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:08:09.816174    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd77893_0ff8_4b2e_95fa_7dca11d72330.slice/crio-bf3b108eb90b2be7d55fb80e44935e2ce3ab0ca7a1c450edf6eb1b1e7ae04d47 WatchSource:0}: Error finding container bf3b108eb90b2be7d55fb80e44935e2ce3ab0ca7a1c450edf6eb1b1e7ae04d47: Status 404 returned error can't find the container with id bf3b108eb90b2be7d55fb80e44935e2ce3ab0ca7a1c450edf6eb1b1e7ae04d47
Sep 18 19:08:09.818634 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.818603    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:08:09.818754 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.818684    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:08:09.818828 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:09.818750    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:08:10.161974 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:10.161877    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" event={"ID":"afd77893-0ff8-4b2e-95fa-7dca11d72330","Type":"ContainerStarted","Data":"70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104"}
Sep 18 19:08:10.161974 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:10.161955    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" event={"ID":"afd77893-0ff8-4b2e-95fa-7dca11d72330","Type":"ContainerStarted","Data":"bf3b108eb90b2be7d55fb80e44935e2ce3ab0ca7a1c450edf6eb1b1e7ae04d47"}
Sep 18 19:08:10.188476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:10.188419    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" podStartSLOduration=1.188401772 podStartE2EDuration="1.188401772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:08:10.186088522 +0000 UTC m=+1426.405096112" watchObservedRunningTime="2026-09-18 19:08:10.188401772 +0000 UTC m=+1426.407409380"
Sep 18 19:08:10.496340 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:10.496300    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:10.501989 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:10.501958    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:11.167875 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:11.167821    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:11.168876 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:11.168845    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:15.001341 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.001305    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"]
Sep 18 19:08:15.005113 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.005091    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.009343 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.009315    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-942883\"/\"distinct-w4-942883-kserve-self-signed-certs\""
Sep 18 19:08:15.026169 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.026124    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"]
Sep 18 19:08:15.068803 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.068768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tmp-dir\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.068988 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.068815    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tls-certs\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.068988 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.068906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-home\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.068988 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.068945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-dshm\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.068988 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.068981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-model-cache\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.069135 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.068998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjrkx\" (UniqueName: \"kubernetes.io/projected/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-kube-api-access-kjrkx\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.170109 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.170067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-home\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.170289 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.170136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-dshm\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.170289 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.170170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-model-cache\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.170289 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.170189    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjrkx\" (UniqueName: \"kubernetes.io/projected/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-kube-api-access-kjrkx\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.170289 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.170251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tmp-dir\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.170289 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.170274    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tls-certs\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.170558 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.170533    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-home\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.170612 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.170573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-model-cache\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.170664 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.170625    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tmp-dir\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.172880 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.172846    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-dshm\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.173013 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.172996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tls-certs\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.182313 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.182276    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjrkx\" (UniqueName: \"kubernetes.io/projected/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-kube-api-access-kjrkx\") pod \"distinct-w4-942883-kserve-6b99c9cd49-m9d2f\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") " pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.316147 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.316018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:15.647089 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:15.647056    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"]
Sep 18 19:08:15.648517 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:08:15.648483    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod934ae89f_9ad4_48c7_a611_fd1d64b3b47a.slice/crio-e041e033b3705c865707823696712e85772f094ef4761ca41577fdc7b184186c WatchSource:0}: Error finding container e041e033b3705c865707823696712e85772f094ef4761ca41577fdc7b184186c: Status 404 returned error can't find the container with id e041e033b3705c865707823696712e85772f094ef4761ca41577fdc7b184186c
Sep 18 19:08:16.210135 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:16.210098    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f" event={"ID":"934ae89f-9ad4-48c7-a611-fd1d64b3b47a","Type":"ContainerStarted","Data":"f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5"}
Sep 18 19:08:16.210575 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:16.210145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f" event={"ID":"934ae89f-9ad4-48c7-a611-fd1d64b3b47a","Type":"ContainerStarted","Data":"e041e033b3705c865707823696712e85772f094ef4761ca41577fdc7b184186c"}
Sep 18 19:08:16.234568 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:16.234502    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f" podStartSLOduration=2.2344826 podStartE2EDuration="2.2344826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:08:16.23245248 +0000 UTC m=+1432.451460057" watchObservedRunningTime="2026-09-18 19:08:16.2344826 +0000 UTC m=+1432.453490178"
Sep 18 19:08:25.316830 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:25.316789    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:25.317254 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:25.316845    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:25.331314 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:25.331280    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:26.285905 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:26.285875    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:08:38.353597 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:38.353500    2581 generic.go:356] "Generic (PLEG): container finished" podID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerID="70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104" exitCode=0
Sep 18 19:08:38.353597 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:38.353573    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" event={"ID":"afd77893-0ff8-4b2e-95fa-7dca11d72330","Type":"ContainerDied","Data":"70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104"}
Sep 18 19:08:38.354090 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:38.354058    2581 scope.go:117] "RemoveContainer" containerID="70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104"
Sep 18 19:08:38.355310 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:38.355286    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:08:38.355377 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:38.355350    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:08:38.355413 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:38.355385    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:08:39.362946 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:39.362913    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" event={"ID":"afd77893-0ff8-4b2e-95fa-7dca11d72330","Type":"ContainerStarted","Data":"d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6"}
Sep 18 19:08:39.496555 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:39.496517    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:39.497668 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:39.497643    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 18 19:08:39.497812 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:39.497695    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 18 19:08:40.496370 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:40.496333    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 18 19:08:40.496873 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:40.496391    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 18 19:08:41.168839 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:41.168800    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:41.497016 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:41.496930    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 18 19:08:41.497016 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:41.496991    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 18 19:08:42.496507 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:42.496470    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 18 19:08:42.496689 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:42.496525    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 18 19:08:43.496170 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:43.496131    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 18 19:08:43.496615 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:43.496187    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 18 19:08:44.496860 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:44.496817    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 18 19:08:44.497256 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:44.496890    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 18 19:08:45.501965 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:45.501934    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:45.503210 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:45.503185    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:08:49.301308 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.301275    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"]
Sep 18 19:08:49.301776 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.301503    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d" podUID="881a3d78-9d3b-430d-b30d-935a73344ff0" containerName="maas-api" containerID="cri-o://e2ac3779006edecfa36ba541223fe27ac412dbc1e0bb1c6748f2db221ea9fb28" gracePeriod=30
Sep 18 19:08:49.435418 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.435376    2581 generic.go:356] "Generic (PLEG): container finished" podID="881a3d78-9d3b-430d-b30d-935a73344ff0" containerID="e2ac3779006edecfa36ba541223fe27ac412dbc1e0bb1c6748f2db221ea9fb28" exitCode=0
Sep 18 19:08:49.435619 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.435577    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d" event={"ID":"881a3d78-9d3b-430d-b30d-935a73344ff0","Type":"ContainerDied","Data":"e2ac3779006edecfa36ba541223fe27ac412dbc1e0bb1c6748f2db221ea9fb28"}
Sep 18 19:08:49.575722 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.575674    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:08:49.654155 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.653691    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-metrics-certs\") pod \"881a3d78-9d3b-430d-b30d-935a73344ff0\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") "
Sep 18 19:08:49.654882 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.654859    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww82v\" (UniqueName: \"kubernetes.io/projected/881a3d78-9d3b-430d-b30d-935a73344ff0-kube-api-access-ww82v\") pod \"881a3d78-9d3b-430d-b30d-935a73344ff0\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") "
Sep 18 19:08:49.655143 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.655129    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-maas-api-tls\") pod \"881a3d78-9d3b-430d-b30d-935a73344ff0\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") "
Sep 18 19:08:49.656137 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.655657    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/881a3d78-9d3b-430d-b30d-935a73344ff0-openshift-service-ca\") pod \"881a3d78-9d3b-430d-b30d-935a73344ff0\" (UID: \"881a3d78-9d3b-430d-b30d-935a73344ff0\") "
Sep 18 19:08:49.656368 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.656098    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/881a3d78-9d3b-430d-b30d-935a73344ff0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "881a3d78-9d3b-430d-b30d-935a73344ff0" (UID: "881a3d78-9d3b-430d-b30d-935a73344ff0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:08:49.656906 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.656889    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/881a3d78-9d3b-430d-b30d-935a73344ff0-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:08:49.657979 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.657953    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "881a3d78-9d3b-430d-b30d-935a73344ff0" (UID: "881a3d78-9d3b-430d-b30d-935a73344ff0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:08:49.659919 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.659884    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881a3d78-9d3b-430d-b30d-935a73344ff0-kube-api-access-ww82v" (OuterVolumeSpecName: "kube-api-access-ww82v") pod "881a3d78-9d3b-430d-b30d-935a73344ff0" (UID: "881a3d78-9d3b-430d-b30d-935a73344ff0"). InnerVolumeSpecName "kube-api-access-ww82v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:08:49.662264 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.662228    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "881a3d78-9d3b-430d-b30d-935a73344ff0" (UID: "881a3d78-9d3b-430d-b30d-935a73344ff0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:08:49.757840 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.757794    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww82v\" (UniqueName: \"kubernetes.io/projected/881a3d78-9d3b-430d-b30d-935a73344ff0-kube-api-access-ww82v\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:08:49.757840 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.757843    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:08:49.758054 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:49.757860    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881a3d78-9d3b-430d-b30d-935a73344ff0-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:08:50.443446 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:50.443413    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"
Sep 18 19:08:50.443919 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:50.443444    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d" event={"ID":"881a3d78-9d3b-430d-b30d-935a73344ff0","Type":"ContainerDied","Data":"6b09ec785eafdba4ffaf5ae073cbda196baf66c68f24ff1ab0836b289d7894d3"}
Sep 18 19:08:50.443919 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:50.443508    2581 scope.go:117] "RemoveContainer" containerID="e2ac3779006edecfa36ba541223fe27ac412dbc1e0bb1c6748f2db221ea9fb28"
Sep 18 19:08:50.484048 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:50.484014    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"]
Sep 18 19:08:50.491901 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:50.491860    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ac9ac7d-7cd88d6c4c-kmn6d"]
Sep 18 19:08:52.315309 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:08:52.315277    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="881a3d78-9d3b-430d-b30d-935a73344ff0" path="/var/lib/kubelet/pods/881a3d78-9d3b-430d-b30d-935a73344ff0/volumes"
Sep 18 19:09:26.387837 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:26.387801    2581 scope.go:117] "RemoveContainer" containerID="f8dd8c6234403904281638a2c41572c297711f02f9af4dca65b93c5ac9a928eb"
Sep 18 19:09:59.137419 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.137379    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"]
Sep 18 19:09:59.138147 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.137631    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy" containerID="cri-o://02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a" gracePeriod=30
Sep 18 19:09:59.293246 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.293209    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"]
Sep 18 19:09:59.293743 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.293693    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881a3d78-9d3b-430d-b30d-935a73344ff0" containerName="maas-api"
Sep 18 19:09:59.293743 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.293734    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="881a3d78-9d3b-430d-b30d-935a73344ff0" containerName="maas-api"
Sep 18 19:09:59.293910 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.293894    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="881a3d78-9d3b-430d-b30d-935a73344ff0" containerName="maas-api"
Sep 18 19:09:59.297055 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.297033    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.300199 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.300172    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1a444c\"/\"embedding-w0-1a444c-kserve-self-signed-certs\""
Sep 18 19:09:59.312174 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.312114    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"]
Sep 18 19:09:59.405190 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.405078    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-model-cache\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.405377 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.405244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-tmp-dir\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.405444 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.405396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-dshm\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.405534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.405508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-home\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.405675 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.405572    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kcg4\" (UniqueName: \"kubernetes.io/projected/53e5e98a-64ca-4a69-a924-200c640ef380-kube-api-access-5kcg4\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.405675 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.405610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e5e98a-64ca-4a69-a924-200c640ef380-tls-certs\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.506413 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.506372    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-home\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.506413 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.506432    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kcg4\" (UniqueName: \"kubernetes.io/projected/53e5e98a-64ca-4a69-a924-200c640ef380-kube-api-access-5kcg4\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.506740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.506457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e5e98a-64ca-4a69-a924-200c640ef380-tls-certs\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.506740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.506504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-model-cache\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.506740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.506545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-tmp-dir\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.506740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.506578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-dshm\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.507006 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.506908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-home\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.507405 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.507374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-tmp-dir\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.507591 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.507567    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-model-cache\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.509154 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.509125    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-dshm\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.510174 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.510149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e5e98a-64ca-4a69-a924-200c640ef380-tls-certs\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.529123 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.529082    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kcg4\" (UniqueName: \"kubernetes.io/projected/53e5e98a-64ca-4a69-a924-200c640ef380-kube-api-access-5kcg4\") pod \"embedding-w0-1a444c-kserve-65fb8578f-7gqk5\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") " pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.607952 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.607912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:09:59.857857 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.857810    2581 patch_prober.go:28] interesting pod/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 19:09:59.858052 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.857886    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 19:09:59.915056 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:09:59.915017    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"]
Sep 18 19:09:59.915513 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:09:59.915478    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53e5e98a_64ca_4a69_a924_200c640ef380.slice/crio-bfdfb9ea68d5acc86157f2af5c1cb5a13f42fb29019fff2408e4d5ddcd8f5239 WatchSource:0}: Error finding container bfdfb9ea68d5acc86157f2af5c1cb5a13f42fb29019fff2408e4d5ddcd8f5239: Status 404 returned error can't find the container with id bfdfb9ea68d5acc86157f2af5c1cb5a13f42fb29019fff2408e4d5ddcd8f5239
Sep 18 19:10:00.888315 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:00.888281    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5" event={"ID":"53e5e98a-64ca-4a69-a924-200c640ef380","Type":"ContainerStarted","Data":"0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894"}
Sep 18 19:10:00.888315 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:00.888323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5" event={"ID":"53e5e98a-64ca-4a69-a924-200c640ef380","Type":"ContainerStarted","Data":"bfdfb9ea68d5acc86157f2af5c1cb5a13f42fb29019fff2408e4d5ddcd8f5239"}
Sep 18 19:10:00.921610 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:00.921556    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5" podStartSLOduration=1.921533371 podStartE2EDuration="1.921533371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:09:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:10:00.920958236 +0000 UTC m=+1537.139965813" watchObservedRunningTime="2026-09-18 19:10:00.921533371 +0000 UTC m=+1537.140540948"
Sep 18 19:10:04.399502 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.399469    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:10:04.565845 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.565795    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-podinfo\") pod \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") "
Sep 18 19:10:04.566163 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.565863    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-certs\") pod \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") "
Sep 18 19:10:04.566163 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.565882    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d87rm\" (UniqueName: \"kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-kube-api-access-d87rm\") pod \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") "
Sep 18 19:10:04.566163 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.565902    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-socket\") pod \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") "
Sep 18 19:10:04.566163 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.565936    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-token\") pod \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") "
Sep 18 19:10:04.566163 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.565962    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-envoy\") pod \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") "
Sep 18 19:10:04.566163 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566050    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-data\") pod \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") "
Sep 18 19:10:04.566163 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566121    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istiod-ca-cert\") pod \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") "
Sep 18 19:10:04.566163 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566161    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-credential-socket\") pod \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\" (UID: \"44613b45-0523-4bb5-ac8e-0fec3d7cb093\") "
Sep 18 19:10:04.566582 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566245    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44613b45-0523-4bb5-ac8e-0fec3d7cb093" (UID: "44613b45-0523-4bb5-ac8e-0fec3d7cb093"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:10:04.566582 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566258    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44613b45-0523-4bb5-ac8e-0fec3d7cb093" (UID: "44613b45-0523-4bb5-ac8e-0fec3d7cb093"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:10:04.566582 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566468    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-data" (OuterVolumeSpecName: "istio-data") pod "44613b45-0523-4bb5-ac8e-0fec3d7cb093" (UID: "44613b45-0523-4bb5-ac8e-0fec3d7cb093"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:10:04.566582 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566568    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44613b45-0523-4bb5-ac8e-0fec3d7cb093" (UID: "44613b45-0523-4bb5-ac8e-0fec3d7cb093"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:10:04.567010 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566625    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:10:04.567010 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566674    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:10:04.567010 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566864    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:10:04.567010 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.566915    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44613b45-0523-4bb5-ac8e-0fec3d7cb093" (UID: "44613b45-0523-4bb5-ac8e-0fec3d7cb093"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:10:04.568427 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.568398    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-kube-api-access-d87rm" (OuterVolumeSpecName: "kube-api-access-d87rm") pod "44613b45-0523-4bb5-ac8e-0fec3d7cb093" (UID: "44613b45-0523-4bb5-ac8e-0fec3d7cb093"). InnerVolumeSpecName "kube-api-access-d87rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:10:04.568546 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.568444    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44613b45-0523-4bb5-ac8e-0fec3d7cb093" (UID: "44613b45-0523-4bb5-ac8e-0fec3d7cb093"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:10:04.568546 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.568453    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44613b45-0523-4bb5-ac8e-0fec3d7cb093" (UID: "44613b45-0523-4bb5-ac8e-0fec3d7cb093"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:10:04.568546 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.568508    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-token" (OuterVolumeSpecName: "istio-token") pod "44613b45-0523-4bb5-ac8e-0fec3d7cb093" (UID: "44613b45-0523-4bb5-ac8e-0fec3d7cb093"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:10:04.668261 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.668207    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d87rm\" (UniqueName: \"kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-kube-api-access-d87rm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:10:04.668261 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.668251    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:10:04.668261 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.668267    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:10:04.668525 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.668277    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:10:04.668525 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.668289    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44613b45-0523-4bb5-ac8e-0fec3d7cb093-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:10:04.668525 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.668300    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44613b45-0523-4bb5-ac8e-0fec3d7cb093-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:10:04.913725 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.913615    2581 generic.go:356] "Generic (PLEG): container finished" podID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerID="02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a" exitCode=0
Sep 18 19:10:04.913725 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.913666    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"
Sep 18 19:10:04.913725 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.913696    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" event={"ID":"44613b45-0523-4bb5-ac8e-0fec3d7cb093","Type":"ContainerDied","Data":"02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a"}
Sep 18 19:10:04.913978 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.913775    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m" event={"ID":"44613b45-0523-4bb5-ac8e-0fec3d7cb093","Type":"ContainerDied","Data":"34e34e01ea4836178a239a037dd35c692f861fe508513eb15b86968d1e6c634b"}
Sep 18 19:10:04.913978 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.913806    2581 scope.go:117] "RemoveContainer" containerID="02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a"
Sep 18 19:10:04.924582 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.924557    2581 scope.go:117] "RemoveContainer" containerID="0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9"
Sep 18 19:10:04.935176 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.935154    2581 scope.go:117] "RemoveContainer" containerID="02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a"
Sep 18 19:10:04.935572 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:10:04.935548    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a\": container with ID starting with 02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a not found: ID does not exist" containerID="02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a"
Sep 18 19:10:04.935689 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.935588    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a"} err="failed to get container status \"02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a\": rpc error: code = NotFound desc = could not find container \"02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a\": container with ID starting with 02ab85f1aa183130d378749bb16843871c4655f50806eb628c5a6b0cb3ca661a not found: ID does not exist"
Sep 18 19:10:04.935689 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.935614    2581 scope.go:117] "RemoveContainer" containerID="0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9"
Sep 18 19:10:04.935927 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:10:04.935907    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9\": container with ID starting with 0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9 not found: ID does not exist" containerID="0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9"
Sep 18 19:10:04.935986 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.935935    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9"} err="failed to get container status \"0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9\": rpc error: code = NotFound desc = could not find container \"0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9\": container with ID starting with 0c50fc76c1a06f5c0071438ea0e4a381473dd9cfb8203311718c1cb79fe32da9 not found: ID does not exist"
Sep 18 19:10:04.967238 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.967207    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"]
Sep 18 19:10:04.969610 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:04.969587    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4ac9ac7d-openshift-default-b4b5986b8-ltq7m"]
Sep 18 19:10:06.314423 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:06.314384    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" path="/var/lib/kubelet/pods/44613b45-0523-4bb5-ac8e-0fec3d7cb093/volumes"
Sep 18 19:10:09.608117 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:09.608061    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:10:09.608117 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:09.608127    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:10:09.621354 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:09.621320    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:10:09.956337 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:10:09.956307    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:11:00.253541 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:00.253496    2581 generic.go:356] "Generic (PLEG): container finished" podID="7673227a-1079-400b-a86d-83de486404ad" containerID="fc2c1f255eae53ca09f821e85fb4cca907152e1ee1abc6f8e16ecc32fcef160b" exitCode=137
Sep 18 19:11:00.254184 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:00.253579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x" event={"ID":"7673227a-1079-400b-a86d-83de486404ad","Type":"ContainerDied","Data":"fc2c1f255eae53ca09f821e85fb4cca907152e1ee1abc6f8e16ecc32fcef160b"}
Sep 18 19:11:00.254184 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:00.254029    2581 scope.go:117] "RemoveContainer" containerID="fc2c1f255eae53ca09f821e85fb4cca907152e1ee1abc6f8e16ecc32fcef160b"
Sep 18 19:11:01.262669 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:01.262632    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x" event={"ID":"7673227a-1079-400b-a86d-83de486404ad","Type":"ContainerStarted","Data":"30742048d4905be0ad2d966600dc6d70eadd2ef7359efa9595b685ecb9ebf602"}
Sep 18 19:11:01.263125 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:01.262889    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 19:11:12.272901 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:12.272860    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 19:11:14.325216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.325180    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"]
Sep 18 19:11:14.325681 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.325609    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy"
Sep 18 19:11:14.325681 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.325621    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy"
Sep 18 19:11:14.325681 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.325645    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy"
Sep 18 19:11:14.325681 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.325651    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy"
Sep 18 19:11:14.325970 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.325738    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy"
Sep 18 19:11:14.329091 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.329063    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.336214 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.336182    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-1d6196-serving-cert\""
Sep 18 19:11:14.355979 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.355943    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"]
Sep 18 19:11:14.501267 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.501228    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cwss\" (UniqueName: \"kubernetes.io/projected/4e58129a-b929-4f51-baad-28acfc5098b2-kube-api-access-4cwss\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.501267 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.501277    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-metrics-certs\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.501515 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.501317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e58129a-b929-4f51-baad-28acfc5098b2-openshift-service-ca\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.501515 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.501392    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-maas-api-tls\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.602385 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.602282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e58129a-b929-4f51-baad-28acfc5098b2-openshift-service-ca\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.602385 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.602335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-maas-api-tls\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.602591 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.602417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cwss\" (UniqueName: \"kubernetes.io/projected/4e58129a-b929-4f51-baad-28acfc5098b2-kube-api-access-4cwss\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.602591 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.602440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-metrics-certs\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.603071 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.603032    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e58129a-b929-4f51-baad-28acfc5098b2-openshift-service-ca\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.605304 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.605275    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-maas-api-tls\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.605423 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.605275    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-metrics-certs\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.618866 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.618831    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cwss\" (UniqueName: \"kubernetes.io/projected/4e58129a-b929-4f51-baad-28acfc5098b2-kube-api-access-4cwss\") pod \"maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.640879 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.640838    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:14.933797 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.933754    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"]
Sep 18 19:11:14.937240 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:11:14.937206    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e58129a_b929_4f51_baad_28acfc5098b2.slice/crio-bd3d0f971e6fa7449355e15c301d5a7042d12cd605bb5c467cc1a2935082ba4b WatchSource:0}: Error finding container bd3d0f971e6fa7449355e15c301d5a7042d12cd605bb5c467cc1a2935082ba4b: Status 404 returned error can't find the container with id bd3d0f971e6fa7449355e15c301d5a7042d12cd605bb5c467cc1a2935082ba4b
Sep 18 19:11:14.938764 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:14.938741    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:11:15.356612 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:15.356454    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69" event={"ID":"4e58129a-b929-4f51-baad-28acfc5098b2","Type":"ContainerStarted","Data":"bd3d0f971e6fa7449355e15c301d5a7042d12cd605bb5c467cc1a2935082ba4b"}
Sep 18 19:11:16.364204 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:16.364157    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69" event={"ID":"4e58129a-b929-4f51-baad-28acfc5098b2","Type":"ContainerStarted","Data":"dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf"}
Sep 18 19:11:16.364639 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:16.364290    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:16.388218 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:16.388154    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69" podStartSLOduration=1.922889082 podStartE2EDuration="2.388135817s" totalImagesPullingTime="465.246735ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:11:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:11:16.385399732 +0000 UTC m=+1612.604407310" watchObservedRunningTime="2026-09-18 19:11:16.388135817 +0000 UTC m=+1612.607143396"
Sep 18 19:11:22.375759 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:22.375696    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:11:40.269436 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.269390    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"]
Sep 18 19:11:40.270312 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.270285    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="44613b45-0523-4bb5-ac8e-0fec3d7cb093" containerName="istio-proxy"
Sep 18 19:11:40.273830 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.273794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.279296 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.279261    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-60d8bd-openshift-default-dockercfg-lwsmq\""
Sep 18 19:11:40.390236 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.390192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-data\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.390463 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.390279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc7kd\" (UniqueName: \"kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-kube-api-access-dc7kd\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.390463 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.390317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-certs\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.390463 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.390345    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-podinfo\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.390605 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.390457    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istiod-ca-cert\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.390605 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.390530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-envoy\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.390605 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.390585    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-token\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.390764 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.390654    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-socket\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.390901 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.390778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-credential-socket\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.412643 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.412603    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"]
Sep 18 19:11:40.491790 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.491746    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-credential-socket\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492006 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.491820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-data\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492006 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.491872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc7kd\" (UniqueName: \"kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-kube-api-access-dc7kd\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492006 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.491905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-certs\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492006 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.491929    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-podinfo\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492006 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.491964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istiod-ca-cert\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492296 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.492010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-envoy\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492296 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.492065    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-token\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492296 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.492117    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-socket\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492296 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.492225    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-credential-socket\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492296 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.492268    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-data\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492507 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.492317    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-certs\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492579 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.492544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-socket\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.492876 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.492855    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istiod-ca-cert\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.494556 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.494529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-envoy\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.495309 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.495053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-podinfo\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.529139 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.528949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-token\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.532008 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.531974    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc7kd\" (UniqueName: \"kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-kube-api-access-dc7kd\") pod \"e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") " pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.584964 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.584919    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:40.889067 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.889036    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"]
Sep 18 19:11:40.890887 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:11:40.890852    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2a9424_cc6a_429f_a3f7_1bfcb7c37102.slice/crio-87177fd96b5c3302cab27adf7cf3d67858b777bf27196f7eaf81c12e6219f3fb WatchSource:0}: Error finding container 87177fd96b5c3302cab27adf7cf3d67858b777bf27196f7eaf81c12e6219f3fb: Status 404 returned error can't find the container with id 87177fd96b5c3302cab27adf7cf3d67858b777bf27196f7eaf81c12e6219f3fb
Sep 18 19:11:40.893336 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.893300    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:11:40.893421 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.893377    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:11:40.893421 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:40.893406    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:11:41.524123 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:41.524088    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" event={"ID":"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102","Type":"ContainerStarted","Data":"edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b"}
Sep 18 19:11:41.524534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:41.524145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" event={"ID":"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102","Type":"ContainerStarted","Data":"87177fd96b5c3302cab27adf7cf3d67858b777bf27196f7eaf81c12e6219f3fb"}
Sep 18 19:11:41.579854 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:41.579773    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" podStartSLOduration=1.579747869 podStartE2EDuration="1.579747869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:11:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:11:41.57650539 +0000 UTC m=+1637.795512968" watchObservedRunningTime="2026-09-18 19:11:41.579747869 +0000 UTC m=+1637.798755440"
Sep 18 19:11:41.585656 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:41.585620    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:41.592594 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:41.592562    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:41.673500 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:41.673457    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"]
Sep 18 19:11:41.685657 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:41.685615    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"]
Sep 18 19:11:41.711223 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:41.711190    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"]
Sep 18 19:11:42.532298 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:42.532261    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:42.533495 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:42.533471    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:11:56.700052 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:56.699978    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r" podUID="64a91386-f39a-4807-a16a-bf86ccd8ec0e" containerName="main" containerID="cri-o://96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d" gracePeriod=45
Sep 18 19:11:56.712481 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:56.712438    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7" podUID="74148608-cbd8-4cc6-a708-7a0bef72f7bb" containerName="main" containerID="cri-o://e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04" gracePeriod=45
Sep 18 19:11:56.732561 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:56.732517    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5" podUID="53e5e98a-64ca-4a69-a924-200c640ef380" containerName="main" containerID="cri-o://0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894" gracePeriod=45
Sep 18 19:11:57.010694 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.010663    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:11:57.039849 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.039819    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:11:57.044381 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.044351    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:11:57.063012 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.062970    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tmp-dir\") pod \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") "
Sep 18 19:11:57.063183 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.063030    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-dshm\") pod \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") "
Sep 18 19:11:57.063183 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.063076    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tls-certs\") pod \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") "
Sep 18 19:11:57.063183 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.063136    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-home\") pod \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") "
Sep 18 19:11:57.063183 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.063164    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngsrp\" (UniqueName: \"kubernetes.io/projected/74148608-cbd8-4cc6-a708-7a0bef72f7bb-kube-api-access-ngsrp\") pod \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") "
Sep 18 19:11:57.063390 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.063207    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-model-cache\") pod \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\" (UID: \"74148608-cbd8-4cc6-a708-7a0bef72f7bb\") "
Sep 18 19:11:57.063390 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.063304    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "74148608-cbd8-4cc6-a708-7a0bef72f7bb" (UID: "74148608-cbd8-4cc6-a708-7a0bef72f7bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.063781 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.063603    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.063781 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.063625    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "74148608-cbd8-4cc6-a708-7a0bef72f7bb" (UID: "74148608-cbd8-4cc6-a708-7a0bef72f7bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.063781 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.063742    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-home" (OuterVolumeSpecName: "home") pod "74148608-cbd8-4cc6-a708-7a0bef72f7bb" (UID: "74148608-cbd8-4cc6-a708-7a0bef72f7bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.065922 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.065888    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74148608-cbd8-4cc6-a708-7a0bef72f7bb-kube-api-access-ngsrp" (OuterVolumeSpecName: "kube-api-access-ngsrp") pod "74148608-cbd8-4cc6-a708-7a0bef72f7bb" (UID: "74148608-cbd8-4cc6-a708-7a0bef72f7bb"). InnerVolumeSpecName "kube-api-access-ngsrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:11:57.065922 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.065900    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "74148608-cbd8-4cc6-a708-7a0bef72f7bb" (UID: "74148608-cbd8-4cc6-a708-7a0bef72f7bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:11:57.066048 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.065937    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-dshm" (OuterVolumeSpecName: "dshm") pod "74148608-cbd8-4cc6-a708-7a0bef72f7bb" (UID: "74148608-cbd8-4cc6-a708-7a0bef72f7bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.164540 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164507    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tmp-dir\") pod \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") "
Sep 18 19:11:57.164776 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164587    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-model-cache\") pod \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") "
Sep 18 19:11:57.164776 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164618    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e5e98a-64ca-4a69-a924-200c640ef380-tls-certs\") pod \"53e5e98a-64ca-4a69-a924-200c640ef380\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") "
Sep 18 19:11:57.164776 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164656    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-home\") pod \"53e5e98a-64ca-4a69-a924-200c640ef380\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") "
Sep 18 19:11:57.164776 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164685    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kcg4\" (UniqueName: \"kubernetes.io/projected/53e5e98a-64ca-4a69-a924-200c640ef380-kube-api-access-5kcg4\") pod \"53e5e98a-64ca-4a69-a924-200c640ef380\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") "
Sep 18 19:11:57.164776 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164744    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-tmp-dir\") pod \"53e5e98a-64ca-4a69-a924-200c640ef380\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") "
Sep 18 19:11:57.165073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164788    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-model-cache\") pod \"53e5e98a-64ca-4a69-a924-200c640ef380\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") "
Sep 18 19:11:57.165073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164815    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tls-certs\") pod \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") "
Sep 18 19:11:57.165073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164840    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8rcp\" (UniqueName: \"kubernetes.io/projected/64a91386-f39a-4807-a16a-bf86ccd8ec0e-kube-api-access-q8rcp\") pod \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") "
Sep 18 19:11:57.165073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164860    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "64a91386-f39a-4807-a16a-bf86ccd8ec0e" (UID: "64a91386-f39a-4807-a16a-bf86ccd8ec0e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.165073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164897    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-home\") pod \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") "
Sep 18 19:11:57.165073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164915    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-model-cache" (OuterVolumeSpecName: "model-cache") pod "64a91386-f39a-4807-a16a-bf86ccd8ec0e" (UID: "64a91386-f39a-4807-a16a-bf86ccd8ec0e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.165073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164936    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-dshm\") pod \"53e5e98a-64ca-4a69-a924-200c640ef380\" (UID: \"53e5e98a-64ca-4a69-a924-200c640ef380\") "
Sep 18 19:11:57.165073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.164999    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-dshm\") pod \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\" (UID: \"64a91386-f39a-4807-a16a-bf86ccd8ec0e\") "
Sep 18 19:11:57.165073 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165061    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "53e5e98a-64ca-4a69-a924-200c640ef380" (UID: "53e5e98a-64ca-4a69-a924-200c640ef380"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165280    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-home" (OuterVolumeSpecName: "home") pod "53e5e98a-64ca-4a69-a924-200c640ef380" (UID: "53e5e98a-64ca-4a69-a924-200c640ef380"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165424    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165442    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165454    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165466    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74148608-cbd8-4cc6-a708-7a0bef72f7bb-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165477    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165490    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165502    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165502    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-home" (OuterVolumeSpecName: "home") pod "64a91386-f39a-4807-a16a-bf86ccd8ec0e" (UID: "64a91386-f39a-4807-a16a-bf86ccd8ec0e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165514    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngsrp\" (UniqueName: \"kubernetes.io/projected/74148608-cbd8-4cc6-a708-7a0bef72f7bb-kube-api-access-ngsrp\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.165602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165525    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74148608-cbd8-4cc6-a708-7a0bef72f7bb-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.166196 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.165739    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-model-cache" (OuterVolumeSpecName: "model-cache") pod "53e5e98a-64ca-4a69-a924-200c640ef380" (UID: "53e5e98a-64ca-4a69-a924-200c640ef380"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.168117 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.168086    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "64a91386-f39a-4807-a16a-bf86ccd8ec0e" (UID: "64a91386-f39a-4807-a16a-bf86ccd8ec0e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:11:57.168325 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.168293    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64a91386-f39a-4807-a16a-bf86ccd8ec0e-kube-api-access-q8rcp" (OuterVolumeSpecName: "kube-api-access-q8rcp") pod "64a91386-f39a-4807-a16a-bf86ccd8ec0e" (UID: "64a91386-f39a-4807-a16a-bf86ccd8ec0e"). InnerVolumeSpecName "kube-api-access-q8rcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:11:57.168579 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.168555    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-dshm" (OuterVolumeSpecName: "dshm") pod "53e5e98a-64ca-4a69-a924-200c640ef380" (UID: "53e5e98a-64ca-4a69-a924-200c640ef380"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.168857 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.168826    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53e5e98a-64ca-4a69-a924-200c640ef380-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "53e5e98a-64ca-4a69-a924-200c640ef380" (UID: "53e5e98a-64ca-4a69-a924-200c640ef380"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:11:57.169143 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.169119    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53e5e98a-64ca-4a69-a924-200c640ef380-kube-api-access-5kcg4" (OuterVolumeSpecName: "kube-api-access-5kcg4") pod "53e5e98a-64ca-4a69-a924-200c640ef380" (UID: "53e5e98a-64ca-4a69-a924-200c640ef380"). InnerVolumeSpecName "kube-api-access-5kcg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:11:57.169688 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.169672    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-dshm" (OuterVolumeSpecName: "dshm") pod "64a91386-f39a-4807-a16a-bf86ccd8ec0e" (UID: "64a91386-f39a-4807-a16a-bf86ccd8ec0e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:11:57.267017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.266911    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8rcp\" (UniqueName: \"kubernetes.io/projected/64a91386-f39a-4807-a16a-bf86ccd8ec0e-kube-api-access-q8rcp\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.267017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.266953    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.267017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.266963    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.267017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.266970    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64a91386-f39a-4807-a16a-bf86ccd8ec0e-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.267017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.266978    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e5e98a-64ca-4a69-a924-200c640ef380-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.267017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.266989    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kcg4\" (UniqueName: \"kubernetes.io/projected/53e5e98a-64ca-4a69-a924-200c640ef380-kube-api-access-5kcg4\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.267017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.266996    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53e5e98a-64ca-4a69-a924-200c640ef380-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.267017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.267004    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64a91386-f39a-4807-a16a-bf86ccd8ec0e-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:11:57.625257 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.625141    2581 generic.go:356] "Generic (PLEG): container finished" podID="74148608-cbd8-4cc6-a708-7a0bef72f7bb" containerID="e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04" exitCode=0
Sep 18 19:11:57.625257 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.625194    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"
Sep 18 19:11:57.625257 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.625241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7" event={"ID":"74148608-cbd8-4cc6-a708-7a0bef72f7bb","Type":"ContainerDied","Data":"e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04"}
Sep 18 19:11:57.625549 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.625296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7" event={"ID":"74148608-cbd8-4cc6-a708-7a0bef72f7bb","Type":"ContainerDied","Data":"8d9b30523308417629e25f1983c47e6067ce256f4d6ba2cb01b94cdd831d5dbe"}
Sep 18 19:11:57.625549 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.625320    2581 scope.go:117] "RemoveContainer" containerID="e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04"
Sep 18 19:11:57.627131 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.627102    2581 generic.go:356] "Generic (PLEG): container finished" podID="53e5e98a-64ca-4a69-a924-200c640ef380" containerID="0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894" exitCode=0
Sep 18 19:11:57.627270 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.627163    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"
Sep 18 19:11:57.627343 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.627271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5" event={"ID":"53e5e98a-64ca-4a69-a924-200c640ef380","Type":"ContainerDied","Data":"0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894"}
Sep 18 19:11:57.627343 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.627306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5" event={"ID":"53e5e98a-64ca-4a69-a924-200c640ef380","Type":"ContainerDied","Data":"bfdfb9ea68d5acc86157f2af5c1cb5a13f42fb29019fff2408e4d5ddcd8f5239"}
Sep 18 19:11:57.629304 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.629160    2581 generic.go:356] "Generic (PLEG): container finished" podID="64a91386-f39a-4807-a16a-bf86ccd8ec0e" containerID="96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d" exitCode=0
Sep 18 19:11:57.629304 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.629198    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r" event={"ID":"64a91386-f39a-4807-a16a-bf86ccd8ec0e","Type":"ContainerDied","Data":"96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d"}
Sep 18 19:11:57.629304 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.629237    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r" event={"ID":"64a91386-f39a-4807-a16a-bf86ccd8ec0e","Type":"ContainerDied","Data":"9227d6f9f7cb27759351a211dfcd19f89446e52d7f419745a75dffee5d32a3fa"}
Sep 18 19:11:57.629629 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.629600    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"
Sep 18 19:11:57.643017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.642993    2581 scope.go:117] "RemoveContainer" containerID="e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04"
Sep 18 19:11:57.643486 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:11:57.643460    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04\": container with ID starting with e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04 not found: ID does not exist" containerID="e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04"
Sep 18 19:11:57.643582 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.643496    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04"} err="failed to get container status \"e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04\": rpc error: code = NotFound desc = could not find container \"e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04\": container with ID starting with e70335b17e7e69c1a88d4a61da0f0a4a96efda36adcfb85017684ea4976a5d04 not found: ID does not exist"
Sep 18 19:11:57.643582 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.643517    2581 scope.go:117] "RemoveContainer" containerID="0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894"
Sep 18 19:11:57.653179 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.653155    2581 scope.go:117] "RemoveContainer" containerID="0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894"
Sep 18 19:11:57.653506 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:11:57.653486    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894\": container with ID starting with 0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894 not found: ID does not exist" containerID="0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894"
Sep 18 19:11:57.653567 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.653517    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894"} err="failed to get container status \"0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894\": rpc error: code = NotFound desc = could not find container \"0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894\": container with ID starting with 0ffec1c48ae2fce458d7786b5c0f246da49d05795dde2993a8f0ac0ca7653894 not found: ID does not exist"
Sep 18 19:11:57.653567 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.653537    2581 scope.go:117] "RemoveContainer" containerID="96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d"
Sep 18 19:11:57.662989 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.662968    2581 scope.go:117] "RemoveContainer" containerID="96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d"
Sep 18 19:11:57.663341 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:11:57.663313    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d\": container with ID starting with 96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d not found: ID does not exist" containerID="96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d"
Sep 18 19:11:57.663445 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.663351    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d"} err="failed to get container status \"96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d\": rpc error: code = NotFound desc = could not find container \"96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d\": container with ID starting with 96930b9c6eec5cbe42ccfc70691ce75396681d319557a7ce30404cca628ae73d not found: ID does not exist"
Sep 18 19:11:57.669165 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.669122    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"]
Sep 18 19:11:57.675004 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.674972    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-w0-1a444c-kserve-7657c4789b-rtzf7"]
Sep 18 19:11:57.689753 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.689723    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"]
Sep 18 19:11:57.697952 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.697922    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/distinct-2-w0-1a444c-kserve-868767d5c5-qx22r"]
Sep 18 19:11:57.714578 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.714552    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"]
Sep 18 19:11:57.727319 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:57.727272    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-1a444c/embedding-w0-1a444c-kserve-65fb8578f-7gqk5"]
Sep 18 19:11:58.313755 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:58.313721    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53e5e98a-64ca-4a69-a924-200c640ef380" path="/var/lib/kubelet/pods/53e5e98a-64ca-4a69-a924-200c640ef380/volumes"
Sep 18 19:11:58.314199 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:58.314184    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64a91386-f39a-4807-a16a-bf86ccd8ec0e" path="/var/lib/kubelet/pods/64a91386-f39a-4807-a16a-bf86ccd8ec0e/volumes"
Sep 18 19:11:58.314558 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:11:58.314546    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74148608-cbd8-4cc6-a708-7a0bef72f7bb" path="/var/lib/kubelet/pods/74148608-cbd8-4cc6-a708-7a0bef72f7bb/volumes"
Sep 18 19:12:07.159823 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:07.159776    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"]
Sep 18 19:12:07.160685 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:07.160627    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy" containerID="cri-o://6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e" gracePeriod=30
Sep 18 19:12:12.444345 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.444314    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 19:12:12.537108 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537072    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-certs\") pod \"7aecaff8-c950-4891-937d-d8a5f87531c2\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") "
Sep 18 19:12:12.537318 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537154    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aecaff8-c950-4891-937d-d8a5f87531c2-istiod-ca-cert\") pod \"7aecaff8-c950-4891-937d-d8a5f87531c2\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") "
Sep 18 19:12:12.537318 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537198    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwxm8\" (UniqueName: \"kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-kube-api-access-hwxm8\") pod \"7aecaff8-c950-4891-937d-d8a5f87531c2\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") "
Sep 18 19:12:12.537318 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537226    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-socket\") pod \"7aecaff8-c950-4891-937d-d8a5f87531c2\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") "
Sep 18 19:12:12.537318 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537267    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-data\") pod \"7aecaff8-c950-4891-937d-d8a5f87531c2\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") "
Sep 18 19:12:12.537318 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537320    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-credential-socket\") pod \"7aecaff8-c950-4891-937d-d8a5f87531c2\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") "
Sep 18 19:12:12.537613 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537391    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-token\") pod \"7aecaff8-c950-4891-937d-d8a5f87531c2\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") "
Sep 18 19:12:12.537613 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537428    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-envoy\") pod \"7aecaff8-c950-4891-937d-d8a5f87531c2\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") "
Sep 18 19:12:12.537613 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537407    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7aecaff8-c950-4891-937d-d8a5f87531c2" (UID: "7aecaff8-c950-4891-937d-d8a5f87531c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:12:12.537613 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537460    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-podinfo\") pod \"7aecaff8-c950-4891-937d-d8a5f87531c2\" (UID: \"7aecaff8-c950-4891-937d-d8a5f87531c2\") "
Sep 18 19:12:12.537863 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537628    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "7aecaff8-c950-4891-937d-d8a5f87531c2" (UID: "7aecaff8-c950-4891-937d-d8a5f87531c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:12:12.537863 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537557    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aecaff8-c950-4891-937d-d8a5f87531c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7aecaff8-c950-4891-937d-d8a5f87531c2" (UID: "7aecaff8-c950-4891-937d-d8a5f87531c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:12:12.537969 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537902    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7aecaff8-c950-4891-937d-d8a5f87531c2" (UID: "7aecaff8-c950-4891-937d-d8a5f87531c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:12:12.537969 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537926    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:12.537969 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537948    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aecaff8-c950-4891-937d-d8a5f87531c2-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:12.537969 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.537963    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:12.538728 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.538541    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7aecaff8-c950-4891-937d-d8a5f87531c2" (UID: "7aecaff8-c950-4891-937d-d8a5f87531c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:12:12.540221 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.540185    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7aecaff8-c950-4891-937d-d8a5f87531c2" (UID: "7aecaff8-c950-4891-937d-d8a5f87531c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:12:12.540383 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.540360    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "7aecaff8-c950-4891-937d-d8a5f87531c2" (UID: "7aecaff8-c950-4891-937d-d8a5f87531c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:12:12.540443 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.540400    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-kube-api-access-hwxm8" (OuterVolumeSpecName: "kube-api-access-hwxm8") pod "7aecaff8-c950-4891-937d-d8a5f87531c2" (UID: "7aecaff8-c950-4891-937d-d8a5f87531c2"). InnerVolumeSpecName "kube-api-access-hwxm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:12:12.540491 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.540469    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7aecaff8-c950-4891-937d-d8a5f87531c2" (UID: "7aecaff8-c950-4891-937d-d8a5f87531c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:12:12.638818 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.638773    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:12.638818 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.638813    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:12.639050 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.638827    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwxm8\" (UniqueName: \"kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-kube-api-access-hwxm8\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:12.639050 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.638842    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:12.639050 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.638855    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aecaff8-c950-4891-937d-d8a5f87531c2-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:12.639050 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.638866    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aecaff8-c950-4891-937d-d8a5f87531c2-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:12.732190 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.732095    2581 generic.go:356] "Generic (PLEG): container finished" podID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerID="6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e" exitCode=0
Sep 18 19:12:12.732190 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.732152    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"
Sep 18 19:12:12.732190 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.732178    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" event={"ID":"7aecaff8-c950-4891-937d-d8a5f87531c2","Type":"ContainerDied","Data":"6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e"}
Sep 18 19:12:12.732445 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.732238    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc" event={"ID":"7aecaff8-c950-4891-937d-d8a5f87531c2","Type":"ContainerDied","Data":"753bbfbf739a9a9c913dd8c58f5453068839785dc05ed70db29d1670de917421"}
Sep 18 19:12:12.732445 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.732262    2581 scope.go:117] "RemoveContainer" containerID="6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e"
Sep 18 19:12:12.743524 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.743506    2581 scope.go:117] "RemoveContainer" containerID="ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f"
Sep 18 19:12:12.754557 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.754537    2581 scope.go:117] "RemoveContainer" containerID="6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e"
Sep 18 19:12:12.754902 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:12:12.754881    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e\": container with ID starting with 6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e not found: ID does not exist" containerID="6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e"
Sep 18 19:12:12.754987 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.754915    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e"} err="failed to get container status \"6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e\": rpc error: code = NotFound desc = could not find container \"6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e\": container with ID starting with 6a0d5884a4838629843db8e638247cdc8000d8b76784cab27301f143f301552e not found: ID does not exist"
Sep 18 19:12:12.754987 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.754939    2581 scope.go:117] "RemoveContainer" containerID="ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f"
Sep 18 19:12:12.755202 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:12:12.755187    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f\": container with ID starting with ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f not found: ID does not exist" containerID="ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f"
Sep 18 19:12:12.755244 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.755209    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f"} err="failed to get container status \"ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f\": rpc error: code = NotFound desc = could not find container \"ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f\": container with ID starting with ed046911077cd9b510a66f9dc13d089e5faabe5fd969f675b8f3cc88356e210f not found: ID does not exist"
Sep 18 19:12:12.779493 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.779450    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"]
Sep 18 19:12:12.797011 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:12.796975    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-1a444c-openshift-default-7bfcdf88d6-w4wgc"]
Sep 18 19:12:13.573921 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:13.573870    2581 patch_prober.go:28] interesting pod/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 19:12:13.574406 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:13.573943    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 19:12:14.314074 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:14.314037    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" path="/var/lib/kubelet/pods/7aecaff8-c950-4891-937d-d8a5f87531c2/volumes"
Sep 18 19:12:15.753864 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:15.753772    2581 generic.go:356] "Generic (PLEG): container finished" podID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerID="edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b" exitCode=0
Sep 18 19:12:15.753864 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:15.753851    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" event={"ID":"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102","Type":"ContainerDied","Data":"edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b"}
Sep 18 19:12:15.754366 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:15.754352    2581 scope.go:117] "RemoveContainer" containerID="edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b"
Sep 18 19:12:15.755676 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:15.755648    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:12:15.755777 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:15.755761    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:12:15.755817 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:15.755796    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:12:16.762385 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:16.762350    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" event={"ID":"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102","Type":"ContainerStarted","Data":"0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618"}
Sep 18 19:12:17.585457 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:17.585404    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:12:17.585682 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:17.585633    2581 patch_prober.go:28] interesting pod/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 18 19:12:17.585803 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:17.585680    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 18 19:12:18.586276 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:18.586229    2581 patch_prober.go:28] interesting pod/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 18 19:12:18.586756 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:18.586300    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 18 19:12:19.585399 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:19.585361    2581 patch_prober.go:28] interesting pod/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 18 19:12:19.585591 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:19.585424    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 18 19:12:20.586067 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:20.586027    2581 patch_prober.go:28] interesting pod/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 18 19:12:20.586491 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:20.586085    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 18 19:12:21.586052 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:21.586014    2581 patch_prober.go:28] interesting pod/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 18 19:12:21.586394 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:21.586358    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 18 19:12:22.590536 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:22.590502    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:12:22.590961 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:22.590833    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:12:22.591763 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:22.591744    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:12:27.995236 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:27.995195    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"]
Sep 18 19:12:27.995733 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:27.995480    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69" podUID="4e58129a-b929-4f51-baad-28acfc5098b2" containerName="maas-api" containerID="cri-o://dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf" gracePeriod=30
Sep 18 19:12:28.258899 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.258826    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:12:28.304651 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.304618    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e58129a-b929-4f51-baad-28acfc5098b2-openshift-service-ca\") pod \"4e58129a-b929-4f51-baad-28acfc5098b2\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") "
Sep 18 19:12:28.304878 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.304716    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-maas-api-tls\") pod \"4e58129a-b929-4f51-baad-28acfc5098b2\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") "
Sep 18 19:12:28.304878 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.304767    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cwss\" (UniqueName: \"kubernetes.io/projected/4e58129a-b929-4f51-baad-28acfc5098b2-kube-api-access-4cwss\") pod \"4e58129a-b929-4f51-baad-28acfc5098b2\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") "
Sep 18 19:12:28.304878 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.304816    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-metrics-certs\") pod \"4e58129a-b929-4f51-baad-28acfc5098b2\" (UID: \"4e58129a-b929-4f51-baad-28acfc5098b2\") "
Sep 18 19:12:28.305120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.305090    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e58129a-b929-4f51-baad-28acfc5098b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4e58129a-b929-4f51-baad-28acfc5098b2" (UID: "4e58129a-b929-4f51-baad-28acfc5098b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:12:28.305254 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.305225    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e58129a-b929-4f51-baad-28acfc5098b2-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:28.307269 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.307239    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4e58129a-b929-4f51-baad-28acfc5098b2" (UID: "4e58129a-b929-4f51-baad-28acfc5098b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:12:28.307492 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.307472    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4e58129a-b929-4f51-baad-28acfc5098b2" (UID: "4e58129a-b929-4f51-baad-28acfc5098b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:12:28.307492 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.307474    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e58129a-b929-4f51-baad-28acfc5098b2-kube-api-access-4cwss" (OuterVolumeSpecName: "kube-api-access-4cwss") pod "4e58129a-b929-4f51-baad-28acfc5098b2" (UID: "4e58129a-b929-4f51-baad-28acfc5098b2"). InnerVolumeSpecName "kube-api-access-4cwss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:12:28.406038 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.405997    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:28.406038 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.406040    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cwss\" (UniqueName: \"kubernetes.io/projected/4e58129a-b929-4f51-baad-28acfc5098b2-kube-api-access-4cwss\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:28.406296 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.406058    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58129a-b929-4f51-baad-28acfc5098b2-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:12:28.840862 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.840831    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e58129a-b929-4f51-baad-28acfc5098b2" containerID="dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf" exitCode=0
Sep 18 19:12:28.841081 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.840880    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"
Sep 18 19:12:28.841081 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.840893    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69" event={"ID":"4e58129a-b929-4f51-baad-28acfc5098b2","Type":"ContainerDied","Data":"dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf"}
Sep 18 19:12:28.841081 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.840947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69" event={"ID":"4e58129a-b929-4f51-baad-28acfc5098b2","Type":"ContainerDied","Data":"bd3d0f971e6fa7449355e15c301d5a7042d12cd605bb5c467cc1a2935082ba4b"}
Sep 18 19:12:28.841081 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.840975    2581 scope.go:117] "RemoveContainer" containerID="dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf"
Sep 18 19:12:28.850946 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.850923    2581 scope.go:117] "RemoveContainer" containerID="dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf"
Sep 18 19:12:28.851319 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:12:28.851290    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf\": container with ID starting with dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf not found: ID does not exist" containerID="dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf"
Sep 18 19:12:28.851397 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.851327    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf"} err="failed to get container status \"dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf\": rpc error: code = NotFound desc = could not find container \"dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf\": container with ID starting with dc290ca19115c9b7c912184c2ea631ba82584755d550c288d6048c842b632ebf not found: ID does not exist"
Sep 18 19:12:28.877216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.877173    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"]
Sep 18 19:12:28.880006 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:28.879973    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6196-77ff75f97f-6mv69"]
Sep 18 19:12:30.314537 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:12:30.314506    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e58129a-b929-4f51-baad-28acfc5098b2" path="/var/lib/kubelet/pods/4e58129a-b929-4f51-baad-28acfc5098b2/volumes"
Sep 18 19:13:04.181971 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:04.181930    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"]
Sep 18 19:13:04.198565 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:04.198528    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"]
Sep 18 19:13:04.589141 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:04.589037    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"]
Sep 18 19:13:04.589322 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:04.589260    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy" containerID="cri-o://0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618" gracePeriod=30
Sep 18 19:13:07.593801 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:07.593750    2581 patch_prober.go:28] interesting pod/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 19:13:07.594281 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:07.593852    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 19:13:09.862204 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.862129    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:13:09.912985 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.912952    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-token\") pod \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") "
Sep 18 19:13:09.913195 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913030    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dc7kd\" (UniqueName: \"kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-kube-api-access-dc7kd\") pod \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") "
Sep 18 19:13:09.913195 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913056    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-credential-socket\") pod \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") "
Sep 18 19:13:09.913195 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913101    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-data\") pod \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") "
Sep 18 19:13:09.913195 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913122    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-socket\") pod \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") "
Sep 18 19:13:09.913195 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913139    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-certs\") pod \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") "
Sep 18 19:13:09.913195 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913160    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-podinfo\") pod \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") "
Sep 18 19:13:09.913195 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913190    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-envoy\") pod \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") "
Sep 18 19:13:09.913579 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913252    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istiod-ca-cert\") pod \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\" (UID: \"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102\") "
Sep 18 19:13:09.913579 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913391    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" (UID: "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:09.913579 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913408    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" (UID: "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:09.913759 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913602    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" (UID: "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:09.913759 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913650    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:09.913759 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913666    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:09.914093 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.913545    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" (UID: "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:09.914093 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.914013    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" (UID: "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:09.915680 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.915641    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-kube-api-access-dc7kd" (OuterVolumeSpecName: "kube-api-access-dc7kd") pod "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" (UID: "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102"). InnerVolumeSpecName "kube-api-access-dc7kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:09.915838 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.915677    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" (UID: "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:09.915915 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.915884    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" (UID: "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:09.916069 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:09.916049    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" (UID: "fe2a9424-cc6a-429f-a3f7-1bfcb7c37102"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:13:10.014209 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.014170    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:10.014209 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.014204    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:10.014209 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.014213    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:10.014209 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.014220    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:10.014484 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.014229    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:10.014484 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.014237    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:10.014484 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.014246    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dc7kd\" (UniqueName: \"kubernetes.io/projected/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102-kube-api-access-dc7kd\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:10.102823 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.102792    2581 generic.go:356] "Generic (PLEG): container finished" podID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerID="0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618" exitCode=0
Sep 18 19:13:10.103051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.102855    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"
Sep 18 19:13:10.103051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.102888    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" event={"ID":"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102","Type":"ContainerDied","Data":"0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618"}
Sep 18 19:13:10.103051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.102961    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm" event={"ID":"fe2a9424-cc6a-429f-a3f7-1bfcb7c37102","Type":"ContainerDied","Data":"87177fd96b5c3302cab27adf7cf3d67858b777bf27196f7eaf81c12e6219f3fb"}
Sep 18 19:13:10.103051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.103002    2581 scope.go:117] "RemoveContainer" containerID="0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618"
Sep 18 19:13:10.113566 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.113546    2581 scope.go:117] "RemoveContainer" containerID="edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b"
Sep 18 19:13:10.123164 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.123133    2581 scope.go:117] "RemoveContainer" containerID="0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618"
Sep 18 19:13:10.123547 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:13:10.123524    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618\": container with ID starting with 0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618 not found: ID does not exist" containerID="0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618"
Sep 18 19:13:10.123610 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.123558    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618"} err="failed to get container status \"0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618\": rpc error: code = NotFound desc = could not find container \"0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618\": container with ID starting with 0184dec3479d0965dd8fb7ecfff8a015d2cff1a81e720b4216f8732625786618 not found: ID does not exist"
Sep 18 19:13:10.123610 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.123578    2581 scope.go:117] "RemoveContainer" containerID="edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b"
Sep 18 19:13:10.123938 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:13:10.123916    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b\": container with ID starting with edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b not found: ID does not exist" containerID="edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b"
Sep 18 19:13:10.124015 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.123946    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b"} err="failed to get container status \"edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b\": rpc error: code = NotFound desc = could not find container \"edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b\": container with ID starting with edb8589ed8376ef4f18252b27c1af8078dd1279739fa9eab9042ce3f69cd3f0b not found: ID does not exist"
Sep 18 19:13:10.135269 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.135235    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"]
Sep 18 19:13:10.146557 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.146518    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-60d8bd-openshift-default-774ff987d6-gh7gm"]
Sep 18 19:13:10.313800 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:10.313765    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" path="/var/lib/kubelet/pods/fe2a9424-cc6a-429f-a3f7-1bfcb7c37102/volumes"
Sep 18 19:13:14.807331 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:14.807298    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"]
Sep 18 19:13:14.807842 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:14.807507    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh" podUID="57af070a-8c05-4025-a32a-c90c13c29f52" containerName="maas-api" containerID="cri-o://1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059" gracePeriod=30
Sep 18 19:13:15.080625 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.080597    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 19:13:15.136937 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.136900    2581 generic.go:356] "Generic (PLEG): container finished" podID="57af070a-8c05-4025-a32a-c90c13c29f52" containerID="1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059" exitCode=0
Sep 18 19:13:15.137121 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.136950    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh" event={"ID":"57af070a-8c05-4025-a32a-c90c13c29f52","Type":"ContainerDied","Data":"1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059"}
Sep 18 19:13:15.137121 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.136970    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"
Sep 18 19:13:15.137121 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.137000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh" event={"ID":"57af070a-8c05-4025-a32a-c90c13c29f52","Type":"ContainerDied","Data":"a7815e760d1c057e063ad3330b023e18f773eceaae7f1676269256e754bf3a5b"}
Sep 18 19:13:15.137121 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.137024    2581 scope.go:117] "RemoveContainer" containerID="1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059"
Sep 18 19:13:15.147650 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.147522    2581 scope.go:117] "RemoveContainer" containerID="1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059"
Sep 18 19:13:15.148146 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:13:15.148074    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059\": container with ID starting with 1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059 not found: ID does not exist" containerID="1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059"
Sep 18 19:13:15.148146 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.148119    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059"} err="failed to get container status \"1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059\": rpc error: code = NotFound desc = could not find container \"1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059\": container with ID starting with 1d5b0464dc7a7de06d41309a4c17854f9d5552601b0c200da0a8f848f0df7059 not found: ID does not exist"
Sep 18 19:13:15.170412 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.170380    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57af070a-8c05-4025-a32a-c90c13c29f52-openshift-service-ca\") pod \"57af070a-8c05-4025-a32a-c90c13c29f52\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") "
Sep 18 19:13:15.170592 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.170455    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-maas-api-tls\") pod \"57af070a-8c05-4025-a32a-c90c13c29f52\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") "
Sep 18 19:13:15.170592 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.170568    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmzp2\" (UniqueName: \"kubernetes.io/projected/57af070a-8c05-4025-a32a-c90c13c29f52-kube-api-access-pmzp2\") pod \"57af070a-8c05-4025-a32a-c90c13c29f52\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") "
Sep 18 19:13:15.170672 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.170656    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-metrics-certs\") pod \"57af070a-8c05-4025-a32a-c90c13c29f52\" (UID: \"57af070a-8c05-4025-a32a-c90c13c29f52\") "
Sep 18 19:13:15.170863 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.170841    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57af070a-8c05-4025-a32a-c90c13c29f52-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57af070a-8c05-4025-a32a-c90c13c29f52" (UID: "57af070a-8c05-4025-a32a-c90c13c29f52"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:15.171027 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.171010    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57af070a-8c05-4025-a32a-c90c13c29f52-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.172873 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.172835    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57af070a-8c05-4025-a32a-c90c13c29f52" (UID: "57af070a-8c05-4025-a32a-c90c13c29f52"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:13:15.173012 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.172868    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "57af070a-8c05-4025-a32a-c90c13c29f52" (UID: "57af070a-8c05-4025-a32a-c90c13c29f52"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:13:15.173012 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.172965    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57af070a-8c05-4025-a32a-c90c13c29f52-kube-api-access-pmzp2" (OuterVolumeSpecName: "kube-api-access-pmzp2") pod "57af070a-8c05-4025-a32a-c90c13c29f52" (UID: "57af070a-8c05-4025-a32a-c90c13c29f52"). InnerVolumeSpecName "kube-api-access-pmzp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:15.269373 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.269337    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"]
Sep 18 19:13:15.269620 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.269575    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z" podUID="d060d921-0a18-469c-847a-b9eb82aa5ae0" containerName="maas-api" containerID="cri-o://4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3" gracePeriod=30
Sep 18 19:13:15.271587 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.271560    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmzp2\" (UniqueName: \"kubernetes.io/projected/57af070a-8c05-4025-a32a-c90c13c29f52-kube-api-access-pmzp2\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.271587 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.271591    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.271826 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.271605    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57af070a-8c05-4025-a32a-c90c13c29f52-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.491269 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.491233    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"]
Sep 18 19:13:15.502751 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.502682    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-942883-6584f6f45b-2r2sh"]
Sep 18 19:13:15.536054 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.536027    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:13:15.676188 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.676158    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppphd\" (UniqueName: \"kubernetes.io/projected/d060d921-0a18-469c-847a-b9eb82aa5ae0-kube-api-access-ppphd\") pod \"d060d921-0a18-469c-847a-b9eb82aa5ae0\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") "
Sep 18 19:13:15.676387 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.676251    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d060d921-0a18-469c-847a-b9eb82aa5ae0-openshift-service-ca\") pod \"d060d921-0a18-469c-847a-b9eb82aa5ae0\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") "
Sep 18 19:13:15.676387 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.676291    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-metrics-certs\") pod \"d060d921-0a18-469c-847a-b9eb82aa5ae0\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") "
Sep 18 19:13:15.676387 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.676369    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-maas-api-tls\") pod \"d060d921-0a18-469c-847a-b9eb82aa5ae0\" (UID: \"d060d921-0a18-469c-847a-b9eb82aa5ae0\") "
Sep 18 19:13:15.676729 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.676666    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d060d921-0a18-469c-847a-b9eb82aa5ae0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d060d921-0a18-469c-847a-b9eb82aa5ae0" (UID: "d060d921-0a18-469c-847a-b9eb82aa5ae0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:15.678759 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.678727    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d060d921-0a18-469c-847a-b9eb82aa5ae0" (UID: "d060d921-0a18-469c-847a-b9eb82aa5ae0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:13:15.679005 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.678983    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d060d921-0a18-469c-847a-b9eb82aa5ae0" (UID: "d060d921-0a18-469c-847a-b9eb82aa5ae0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:13:15.679051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.679021    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d060d921-0a18-469c-847a-b9eb82aa5ae0-kube-api-access-ppphd" (OuterVolumeSpecName: "kube-api-access-ppphd") pod "d060d921-0a18-469c-847a-b9eb82aa5ae0" (UID: "d060d921-0a18-469c-847a-b9eb82aa5ae0"). InnerVolumeSpecName "kube-api-access-ppphd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:15.778497 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.778457    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppphd\" (UniqueName: \"kubernetes.io/projected/d060d921-0a18-469c-847a-b9eb82aa5ae0-kube-api-access-ppphd\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.778497 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.778497    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d060d921-0a18-469c-847a-b9eb82aa5ae0-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.778805 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.778513    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.778805 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:15.778527    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d060d921-0a18-469c-847a-b9eb82aa5ae0-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:16.145057 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.144959    2581 generic.go:356] "Generic (PLEG): container finished" podID="d060d921-0a18-469c-847a-b9eb82aa5ae0" containerID="4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3" exitCode=0
Sep 18 19:13:16.145057 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.145002    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"
Sep 18 19:13:16.145576 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.144998    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z" event={"ID":"d060d921-0a18-469c-847a-b9eb82aa5ae0","Type":"ContainerDied","Data":"4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3"}
Sep 18 19:13:16.145576 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.145169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z" event={"ID":"d060d921-0a18-469c-847a-b9eb82aa5ae0","Type":"ContainerDied","Data":"b619482defe9553e83cf0d72c45295fbcc17e3390d70a73b827c5d254d889a94"}
Sep 18 19:13:16.145576 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.145207    2581 scope.go:117] "RemoveContainer" containerID="4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3"
Sep 18 19:13:16.156556 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.156529    2581 scope.go:117] "RemoveContainer" containerID="4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3"
Sep 18 19:13:16.156930 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:13:16.156909    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3\": container with ID starting with 4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3 not found: ID does not exist" containerID="4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3"
Sep 18 19:13:16.157011 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.156941    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3"} err="failed to get container status \"4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3\": rpc error: code = NotFound desc = could not find container \"4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3\": container with ID starting with 4f8b0f4b36523c7a0f2f2de09c9c3a29ac37aefd1f1f7c0ac184dfdb412d36b3 not found: ID does not exist"
Sep 18 19:13:16.199371 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.199334    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"]
Sep 18 19:13:16.204548 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.204512    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3b5822-66c87ffc59-ntc2z"]
Sep 18 19:13:16.314927 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.314885    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57af070a-8c05-4025-a32a-c90c13c29f52" path="/var/lib/kubelet/pods/57af070a-8c05-4025-a32a-c90c13c29f52/volumes"
Sep 18 19:13:16.315419 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:16.315397    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d060d921-0a18-469c-847a-b9eb82aa5ae0" path="/var/lib/kubelet/pods/d060d921-0a18-469c-847a-b9eb82aa5ae0/volumes"
Sep 18 19:13:19.204186 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.204140    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f" podUID="934ae89f-9ad4-48c7-a611-fd1d64b3b47a" containerName="main" containerID="cri-o://f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5" gracePeriod=45
Sep 18 19:13:19.218577 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.218534    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5" podUID="d0ad3c77-9978-4fe7-845f-449a7846b7c9" containerName="main" containerID="cri-o://c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3" gracePeriod=45
Sep 18 19:13:19.486491 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.486464    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:13:19.510969 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.510941    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:13:19.616304 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616272    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-home\") pod \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") "
Sep 18 19:13:19.616500 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616326    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76d8m\" (UniqueName: \"kubernetes.io/projected/d0ad3c77-9978-4fe7-845f-449a7846b7c9-kube-api-access-76d8m\") pod \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") "
Sep 18 19:13:19.616500 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616365    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-dshm\") pod \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") "
Sep 18 19:13:19.616500 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616399    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tls-certs\") pod \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") "
Sep 18 19:13:19.616500 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616448    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjrkx\" (UniqueName: \"kubernetes.io/projected/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-kube-api-access-kjrkx\") pod \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") "
Sep 18 19:13:19.616500 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616477    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tmp-dir\") pod \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") "
Sep 18 19:13:19.616817 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616501    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-model-cache\") pod \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") "
Sep 18 19:13:19.616817 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616558    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tls-certs\") pod \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") "
Sep 18 19:13:19.616817 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616591    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-model-cache\") pod \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") "
Sep 18 19:13:19.616817 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616620    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-dshm\") pod \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") "
Sep 18 19:13:19.616817 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616623    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-home" (OuterVolumeSpecName: "home") pod "d0ad3c77-9978-4fe7-845f-449a7846b7c9" (UID: "d0ad3c77-9978-4fe7-845f-449a7846b7c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:19.616817 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616651    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tmp-dir\") pod \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\" (UID: \"d0ad3c77-9978-4fe7-845f-449a7846b7c9\") "
Sep 18 19:13:19.616817 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616688    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-home\") pod \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\" (UID: \"934ae89f-9ad4-48c7-a611-fd1d64b3b47a\") "
Sep 18 19:13:19.617343 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.616942    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0ad3c77-9978-4fe7-845f-449a7846b7c9" (UID: "d0ad3c77-9978-4fe7-845f-449a7846b7c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:19.617343 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.617018    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "934ae89f-9ad4-48c7-a611-fd1d64b3b47a" (UID: "934ae89f-9ad4-48c7-a611-fd1d64b3b47a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:19.617343 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.617083    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.617343 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.617102    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.617566 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.617391    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-home" (OuterVolumeSpecName: "home") pod "934ae89f-9ad4-48c7-a611-fd1d64b3b47a" (UID: "934ae89f-9ad4-48c7-a611-fd1d64b3b47a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:19.617801 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.617761    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-model-cache" (OuterVolumeSpecName: "model-cache") pod "934ae89f-9ad4-48c7-a611-fd1d64b3b47a" (UID: "934ae89f-9ad4-48c7-a611-fd1d64b3b47a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:19.618393 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.618342    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0ad3c77-9978-4fe7-845f-449a7846b7c9" (UID: "d0ad3c77-9978-4fe7-845f-449a7846b7c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:19.619429 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.619388    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-dshm" (OuterVolumeSpecName: "dshm") pod "d0ad3c77-9978-4fe7-845f-449a7846b7c9" (UID: "d0ad3c77-9978-4fe7-845f-449a7846b7c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:19.619429 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.619417    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ad3c77-9978-4fe7-845f-449a7846b7c9-kube-api-access-76d8m" (OuterVolumeSpecName: "kube-api-access-76d8m") pod "d0ad3c77-9978-4fe7-845f-449a7846b7c9" (UID: "d0ad3c77-9978-4fe7-845f-449a7846b7c9"). InnerVolumeSpecName "kube-api-access-76d8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:19.620022 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.619947    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-kube-api-access-kjrkx" (OuterVolumeSpecName: "kube-api-access-kjrkx") pod "934ae89f-9ad4-48c7-a611-fd1d64b3b47a" (UID: "934ae89f-9ad4-48c7-a611-fd1d64b3b47a"). InnerVolumeSpecName "kube-api-access-kjrkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:19.620127 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.620043    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0ad3c77-9978-4fe7-845f-449a7846b7c9" (UID: "d0ad3c77-9978-4fe7-845f-449a7846b7c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:13:19.620777 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.620311    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "934ae89f-9ad4-48c7-a611-fd1d64b3b47a" (UID: "934ae89f-9ad4-48c7-a611-fd1d64b3b47a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:13:19.621412 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.621391    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-dshm" (OuterVolumeSpecName: "dshm") pod "934ae89f-9ad4-48c7-a611-fd1d64b3b47a" (UID: "934ae89f-9ad4-48c7-a611-fd1d64b3b47a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:19.717929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.717841    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.717929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.717871    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.717929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.717879    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.717929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.717887    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76d8m\" (UniqueName: \"kubernetes.io/projected/d0ad3c77-9978-4fe7-845f-449a7846b7c9-kube-api-access-76d8m\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.717929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.717897    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ad3c77-9978-4fe7-845f-449a7846b7c9-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.717929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.717907    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.717929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.717915    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjrkx\" (UniqueName: \"kubernetes.io/projected/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-kube-api-access-kjrkx\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.717929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.717922    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.717929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.717929    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/934ae89f-9ad4-48c7-a611-fd1d64b3b47a-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:19.717929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:19.717937    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ad3c77-9978-4fe7-845f-449a7846b7c9-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:20.174013 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.173976    2581 generic.go:356] "Generic (PLEG): container finished" podID="934ae89f-9ad4-48c7-a611-fd1d64b3b47a" containerID="f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5" exitCode=0
Sep 18 19:13:20.174215 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.174019    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"
Sep 18 19:13:20.174215 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.174055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f" event={"ID":"934ae89f-9ad4-48c7-a611-fd1d64b3b47a","Type":"ContainerDied","Data":"f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5"}
Sep 18 19:13:20.174215 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.174103    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f" event={"ID":"934ae89f-9ad4-48c7-a611-fd1d64b3b47a","Type":"ContainerDied","Data":"e041e033b3705c865707823696712e85772f094ef4761ca41577fdc7b184186c"}
Sep 18 19:13:20.174215 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.174126    2581 scope.go:117] "RemoveContainer" containerID="f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5"
Sep 18 19:13:20.176104 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.176081    2581 generic.go:356] "Generic (PLEG): container finished" podID="d0ad3c77-9978-4fe7-845f-449a7846b7c9" containerID="c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3" exitCode=0
Sep 18 19:13:20.176200 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.176120    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"
Sep 18 19:13:20.176200 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.176145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5" event={"ID":"d0ad3c77-9978-4fe7-845f-449a7846b7c9","Type":"ContainerDied","Data":"c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3"}
Sep 18 19:13:20.176200 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.176190    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5" event={"ID":"d0ad3c77-9978-4fe7-845f-449a7846b7c9","Type":"ContainerDied","Data":"aea386b3c62b538685b69247113ea895d6282a4005e4e5853e9d3f716619bd7e"}
Sep 18 19:13:20.185555 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.185534    2581 scope.go:117] "RemoveContainer" containerID="f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5"
Sep 18 19:13:20.185945 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:13:20.185918    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5\": container with ID starting with f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5 not found: ID does not exist" containerID="f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5"
Sep 18 19:13:20.186026 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.185958    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5"} err="failed to get container status \"f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5\": rpc error: code = NotFound desc = could not find container \"f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5\": container with ID starting with f1b9ff172102b183bcb85c4a3f68791ccf522ae06084a0e53c97c60ff0bdc9a5 not found: ID does not exist"
Sep 18 19:13:20.186026 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.185984    2581 scope.go:117] "RemoveContainer" containerID="c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3"
Sep 18 19:13:20.194395 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.194375    2581 scope.go:117] "RemoveContainer" containerID="c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3"
Sep 18 19:13:20.194676 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:13:20.194658    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3\": container with ID starting with c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3 not found: ID does not exist" containerID="c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3"
Sep 18 19:13:20.194774 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.194687    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3"} err="failed to get container status \"c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3\": rpc error: code = NotFound desc = could not find container \"c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3\": container with ID starting with c4cc45f7d41ccbfbd4cb93ba44f52a63c2fd8c6154db302b84f707324d9751c3 not found: ID does not exist"
Sep 18 19:13:20.201616 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.201584    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"]
Sep 18 19:13:20.205498 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.205474    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-942883/distinct-w4-942883-kserve-6b99c9cd49-m9d2f"]
Sep 18 19:13:20.217288 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.217261    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"]
Sep 18 19:13:20.222478 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.222454    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-942883/premium-w4-942883-kserve-6df8d6fb7d-7fjm5"]
Sep 18 19:13:20.316559 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.316526    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="934ae89f-9ad4-48c7-a611-fd1d64b3b47a" path="/var/lib/kubelet/pods/934ae89f-9ad4-48c7-a611-fd1d64b3b47a/volumes"
Sep 18 19:13:20.317066 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:20.317047    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0ad3c77-9978-4fe7-845f-449a7846b7c9" path="/var/lib/kubelet/pods/d0ad3c77-9978-4fe7-845f-449a7846b7c9/volumes"
Sep 18 19:13:37.082743 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.082684    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"]
Sep 18 19:13:37.083236 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.082969    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn" podUID="aef3e227-586e-48a6-a3a0-275230efdbdc" containerName="maas-api" containerID="cri-o://51df87864fa69f77b3784484a095280e675efc1a5b2d0ccb24a7f0f6447189aa" gracePeriod=30
Sep 18 19:13:37.283449 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.283414    2581 generic.go:356] "Generic (PLEG): container finished" podID="aef3e227-586e-48a6-a3a0-275230efdbdc" containerID="51df87864fa69f77b3784484a095280e675efc1a5b2d0ccb24a7f0f6447189aa" exitCode=0
Sep 18 19:13:37.283640 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.283490    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn" event={"ID":"aef3e227-586e-48a6-a3a0-275230efdbdc","Type":"ContainerDied","Data":"51df87864fa69f77b3784484a095280e675efc1a5b2d0ccb24a7f0f6447189aa"}
Sep 18 19:13:37.351581 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.351549    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:13:37.384665 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.384627    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl6tw\" (UniqueName: \"kubernetes.io/projected/aef3e227-586e-48a6-a3a0-275230efdbdc-kube-api-access-sl6tw\") pod \"aef3e227-586e-48a6-a3a0-275230efdbdc\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") "
Sep 18 19:13:37.384665 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.384664    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-maas-api-tls\") pod \"aef3e227-586e-48a6-a3a0-275230efdbdc\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") "
Sep 18 19:13:37.384928 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.384692    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aef3e227-586e-48a6-a3a0-275230efdbdc-openshift-service-ca\") pod \"aef3e227-586e-48a6-a3a0-275230efdbdc\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") "
Sep 18 19:13:37.384928 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.384778    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-metrics-certs\") pod \"aef3e227-586e-48a6-a3a0-275230efdbdc\" (UID: \"aef3e227-586e-48a6-a3a0-275230efdbdc\") "
Sep 18 19:13:37.385154 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.385132    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aef3e227-586e-48a6-a3a0-275230efdbdc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aef3e227-586e-48a6-a3a0-275230efdbdc" (UID: "aef3e227-586e-48a6-a3a0-275230efdbdc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:37.387115 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.387078    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aef3e227-586e-48a6-a3a0-275230efdbdc" (UID: "aef3e227-586e-48a6-a3a0-275230efdbdc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:13:37.387115 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.387094    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aef3e227-586e-48a6-a3a0-275230efdbdc" (UID: "aef3e227-586e-48a6-a3a0-275230efdbdc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:13:37.387286 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.387122    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef3e227-586e-48a6-a3a0-275230efdbdc-kube-api-access-sl6tw" (OuterVolumeSpecName: "kube-api-access-sl6tw") pod "aef3e227-586e-48a6-a3a0-275230efdbdc" (UID: "aef3e227-586e-48a6-a3a0-275230efdbdc"). InnerVolumeSpecName "kube-api-access-sl6tw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:37.486354 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.486301    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:37.486354 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.486347    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl6tw\" (UniqueName: \"kubernetes.io/projected/aef3e227-586e-48a6-a3a0-275230efdbdc-kube-api-access-sl6tw\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:37.486597 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.486363    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aef3e227-586e-48a6-a3a0-275230efdbdc-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:37.486597 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:37.486377    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aef3e227-586e-48a6-a3a0-275230efdbdc-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:38.291393 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:38.291354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn" event={"ID":"aef3e227-586e-48a6-a3a0-275230efdbdc","Type":"ContainerDied","Data":"21da9245afa876b4c582f8b5d269731046104a934ca4bda24646b28466610edd"}
Sep 18 19:13:38.291924 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:38.291413    2581 scope.go:117] "RemoveContainer" containerID="51df87864fa69f77b3784484a095280e675efc1a5b2d0ccb24a7f0f6447189aa"
Sep 18 19:13:38.291924 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:38.291365    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"
Sep 18 19:13:38.322866 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:38.322831    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"]
Sep 18 19:13:38.335371 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:38.335336    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4805a7-867f96969c-2wmbn"]
Sep 18 19:13:40.313680 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:40.313643    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aef3e227-586e-48a6-a3a0-275230efdbdc" path="/var/lib/kubelet/pods/aef3e227-586e-48a6-a3a0-275230efdbdc/volumes"
Sep 18 19:13:46.484419 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:46.484380    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"]
Sep 18 19:13:46.484915 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:46.484598    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy" containerID="cri-o://68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02" gracePeriod=30
Sep 18 19:13:51.738830 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.738793    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:13:51.765413 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765328    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7684-d725da"]
Sep 18 19:13:51.765732 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765697    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aef3e227-586e-48a6-a3a0-275230efdbdc" containerName="maas-api"
Sep 18 19:13:51.765793 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765734    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aef3e227-586e-48a6-a3a0-275230efdbdc" containerName="maas-api"
Sep 18 19:13:51.765793 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765752    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ad3c77-9978-4fe7-845f-449a7846b7c9" containerName="main"
Sep 18 19:13:51.765793 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765759    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ad3c77-9978-4fe7-845f-449a7846b7c9" containerName="main"
Sep 18 19:13:51.765793 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765766    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy"
Sep 18 19:13:51.765793 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765771    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy"
Sep 18 19:13:51.765793 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765781    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53e5e98a-64ca-4a69-a924-200c640ef380" containerName="main"
Sep 18 19:13:51.765793 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765786    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="53e5e98a-64ca-4a69-a924-200c640ef380" containerName="main"
Sep 18 19:13:51.765793 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765792    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74148608-cbd8-4cc6-a708-7a0bef72f7bb" containerName="main"
Sep 18 19:13:51.765793 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765797    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="74148608-cbd8-4cc6-a708-7a0bef72f7bb" containerName="main"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765808    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765813    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765821    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765834    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765840    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d060d921-0a18-469c-847a-b9eb82aa5ae0" containerName="maas-api"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765847    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d060d921-0a18-469c-847a-b9eb82aa5ae0" containerName="maas-api"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765853    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e58129a-b929-4f51-baad-28acfc5098b2" containerName="maas-api"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765858    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e58129a-b929-4f51-baad-28acfc5098b2" containerName="maas-api"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765864    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765869    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765877    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="934ae89f-9ad4-48c7-a611-fd1d64b3b47a" containerName="main"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765883    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="934ae89f-9ad4-48c7-a611-fd1d64b3b47a" containerName="main"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765890    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57af070a-8c05-4025-a32a-c90c13c29f52" containerName="maas-api"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765896    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="57af070a-8c05-4025-a32a-c90c13c29f52" containerName="maas-api"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765903    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64a91386-f39a-4807-a16a-bf86ccd8ec0e" containerName="main"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765908    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="64a91386-f39a-4807-a16a-bf86ccd8ec0e" containerName="main"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765967    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765974    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ad3c77-9978-4fe7-845f-449a7846b7c9" containerName="main"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765979    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="53e5e98a-64ca-4a69-a924-200c640ef380" containerName="main"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765984    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e58129a-b929-4f51-baad-28acfc5098b2" containerName="maas-api"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765990    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aef3e227-586e-48a6-a3a0-275230efdbdc" containerName="maas-api"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.765998    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="64a91386-f39a-4807-a16a-bf86ccd8ec0e" containerName="main"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.766003    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="57af070a-8c05-4025-a32a-c90c13c29f52" containerName="maas-api"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.766008    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d060d921-0a18-469c-847a-b9eb82aa5ae0" containerName="maas-api"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.766013    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.766017    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="74148608-cbd8-4cc6-a708-7a0bef72f7bb" containerName="main"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.766023    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.766030    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.766035    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="934ae89f-9ad4-48c7-a611-fd1d64b3b47a" containerName="main"
Sep 18 19:13:51.766078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.766040    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aecaff8-c950-4891-937d-d8a5f87531c2" containerName="istio-proxy"
Sep 18 19:13:51.769010 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.768989    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7684-d725da"
Sep 18 19:13:51.775304 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.775278    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wgm9b\""
Sep 18 19:13:51.793433 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.793398    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7684-d725da"]
Sep 18 19:13:51.834365 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834324    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-data\") pod \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") "
Sep 18 19:13:51.834534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834383    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-certs\") pod \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") "
Sep 18 19:13:51.834534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834435    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-podinfo\") pod \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") "
Sep 18 19:13:51.834534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834463    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-token\") pod \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") "
Sep 18 19:13:51.834534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834522    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-credential-socket\") pod \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") "
Sep 18 19:13:51.834763 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834562    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qghgh\" (UniqueName: \"kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-kube-api-access-qghgh\") pod \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") "
Sep 18 19:13:51.834763 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834602    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istiod-ca-cert\") pod \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") "
Sep 18 19:13:51.834763 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834637    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-envoy\") pod \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") "
Sep 18 19:13:51.834763 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834646    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0d458cd2-a72f-4c7d-99a6-aa40698fd016" (UID: "0d458cd2-a72f-4c7d-99a6-aa40698fd016"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:51.834763 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834660    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-data" (OuterVolumeSpecName: "istio-data") pod "0d458cd2-a72f-4c7d-99a6-aa40698fd016" (UID: "0d458cd2-a72f-4c7d-99a6-aa40698fd016"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:51.834763 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834694    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-socket\") pod \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\" (UID: \"0d458cd2-a72f-4c7d-99a6-aa40698fd016\") "
Sep 18 19:13:51.835090 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.834953    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0d458cd2-a72f-4c7d-99a6-aa40698fd016" (UID: "0d458cd2-a72f-4c7d-99a6-aa40698fd016"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:51.835090 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.835028    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0d458cd2-a72f-4c7d-99a6-aa40698fd016" (UID: "0d458cd2-a72f-4c7d-99a6-aa40698fd016"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:51.835199 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.835121    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxb6w\" (UniqueName: \"kubernetes.io/projected/a1ae9d3d-c90a-435b-babf-281b67f941c4-kube-api-access-dxb6w\") pod \"test-curl-7684-d725da\" (UID: \"a1ae9d3d-c90a-435b-babf-281b67f941c4\") " pod="opendatahub/test-curl-7684-d725da"
Sep 18 19:13:51.835199 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.835189    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:51.835300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.835206    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:51.835300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.835220    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:51.835300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.835234    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:51.835453 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.835356    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0d458cd2-a72f-4c7d-99a6-aa40698fd016" (UID: "0d458cd2-a72f-4c7d-99a6-aa40698fd016"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:51.837065 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.837035    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0d458cd2-a72f-4c7d-99a6-aa40698fd016" (UID: "0d458cd2-a72f-4c7d-99a6-aa40698fd016"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:51.837210 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.837124    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-token" (OuterVolumeSpecName: "istio-token") pod "0d458cd2-a72f-4c7d-99a6-aa40698fd016" (UID: "0d458cd2-a72f-4c7d-99a6-aa40698fd016"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:51.837282 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.837227    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0d458cd2-a72f-4c7d-99a6-aa40698fd016" (UID: "0d458cd2-a72f-4c7d-99a6-aa40698fd016"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:13:51.837585 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.837560    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-kube-api-access-qghgh" (OuterVolumeSpecName: "kube-api-access-qghgh") pod "0d458cd2-a72f-4c7d-99a6-aa40698fd016" (UID: "0d458cd2-a72f-4c7d-99a6-aa40698fd016"). InnerVolumeSpecName "kube-api-access-qghgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:51.936112 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.936073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxb6w\" (UniqueName: \"kubernetes.io/projected/a1ae9d3d-c90a-435b-babf-281b67f941c4-kube-api-access-dxb6w\") pod \"test-curl-7684-d725da\" (UID: \"a1ae9d3d-c90a-435b-babf-281b67f941c4\") " pod="opendatahub/test-curl-7684-d725da"
Sep 18 19:13:51.936300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.936169    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:51.936300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.936180    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qghgh\" (UniqueName: \"kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-kube-api-access-qghgh\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:51.936300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.936190    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:51.936300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.936201    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:51.936300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.936208    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d458cd2-a72f-4c7d-99a6-aa40698fd016-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:51.950476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:51.950441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxb6w\" (UniqueName: \"kubernetes.io/projected/a1ae9d3d-c90a-435b-babf-281b67f941c4-kube-api-access-dxb6w\") pod \"test-curl-7684-d725da\" (UID: \"a1ae9d3d-c90a-435b-babf-281b67f941c4\") " pod="opendatahub/test-curl-7684-d725da"
Sep 18 19:13:52.077941 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.077847    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7684-d725da"
Sep 18 19:13:52.363983 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.363953    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7684-d725da"]
Sep 18 19:13:52.366004 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:13:52.365973    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1ae9d3d_c90a_435b_babf_281b67f941c4.slice/crio-2868b15fd5a4b741e3da3a311f905336e9a73e97d461ff8fe3508e7244aea865 WatchSource:0}: Error finding container 2868b15fd5a4b741e3da3a311f905336e9a73e97d461ff8fe3508e7244aea865: Status 404 returned error can't find the container with id 2868b15fd5a4b741e3da3a311f905336e9a73e97d461ff8fe3508e7244aea865
Sep 18 19:13:52.374250 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.374225    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerID="68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02" exitCode=0
Sep 18 19:13:52.374335 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.374297    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"
Sep 18 19:13:52.374335 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.374317    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" event={"ID":"0d458cd2-a72f-4c7d-99a6-aa40698fd016","Type":"ContainerDied","Data":"68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02"}
Sep 18 19:13:52.374451 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.374387    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k" event={"ID":"0d458cd2-a72f-4c7d-99a6-aa40698fd016","Type":"ContainerDied","Data":"05b05992f4af3fb3c56db6c510f22de029f479a0dfe8c51b255f0b65fa773ddb"}
Sep 18 19:13:52.374494 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.374466    2581 scope.go:117] "RemoveContainer" containerID="68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02"
Sep 18 19:13:52.384027 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.384006    2581 scope.go:117] "RemoveContainer" containerID="e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2"
Sep 18 19:13:52.395520 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.395489    2581 scope.go:117] "RemoveContainer" containerID="68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02"
Sep 18 19:13:52.396043 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:13:52.396017    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02\": container with ID starting with 68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02 not found: ID does not exist" containerID="68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02"
Sep 18 19:13:52.396182 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.396051    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02"} err="failed to get container status \"68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02\": rpc error: code = NotFound desc = could not find container \"68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02\": container with ID starting with 68713d0ec63d1a64d0a7d11c95cb81fe3ef2ebc3cbbec29a77921b7a668dde02 not found: ID does not exist"
Sep 18 19:13:52.396182 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.396071    2581 scope.go:117] "RemoveContainer" containerID="e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2"
Sep 18 19:13:52.396535 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:13:52.396517    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2\": container with ID starting with e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2 not found: ID does not exist" containerID="e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2"
Sep 18 19:13:52.396583 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.396539    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2"} err="failed to get container status \"e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2\": rpc error: code = NotFound desc = could not find container \"e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2\": container with ID starting with e24deb0524bcbf0f61d2d62eae7c48c291d19d7b26ce9fe8c4b2477384e147e2 not found: ID does not exist"
Sep 18 19:13:52.405387 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.405329    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"]
Sep 18 19:13:52.407812 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:52.407782    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4805a7-openshift-default-65f86d8c78-9kv8k"]
Sep 18 19:13:53.381783 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:53.381734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7684-d725da" event={"ID":"a1ae9d3d-c90a-435b-babf-281b67f941c4","Type":"ContainerStarted","Data":"205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b"}
Sep 18 19:13:53.381783 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:53.381788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7684-d725da" event={"ID":"a1ae9d3d-c90a-435b-babf-281b67f941c4","Type":"ContainerStarted","Data":"2868b15fd5a4b741e3da3a311f905336e9a73e97d461ff8fe3508e7244aea865"}
Sep 18 19:13:53.403022 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:53.402966    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7684-d725da" podStartSLOduration=2.402950744 podStartE2EDuration="2.402950744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:13:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:13:53.402109264 +0000 UTC m=+1769.621116843" watchObservedRunningTime="2026-09-18 19:13:53.402950744 +0000 UTC m=+1769.621958322"
Sep 18 19:13:54.109435 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:54.109341    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7684-d725da"]
Sep 18 19:13:54.123123 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:54.123091    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7684-d725da"]
Sep 18 19:13:54.313611 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:54.313566    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" path="/var/lib/kubelet/pods/0d458cd2-a72f-4c7d-99a6-aa40698fd016/volumes"
Sep 18 19:13:54.388987 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:13:54.388906    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7684-d725da" container="test-curl-7684-d725da"
Sep 18 19:13:55.394215 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:55.394174    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7684-d725da" podUID="a1ae9d3d-c90a-435b-babf-281b67f941c4" containerName="test-curl-7684-d725da" containerID="cri-o://205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b" gracePeriod=2
Sep 18 19:13:57.577514 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:57.577485    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7684-d725da"
Sep 18 19:13:57.686637 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:57.686594    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxb6w\" (UniqueName: \"kubernetes.io/projected/a1ae9d3d-c90a-435b-babf-281b67f941c4-kube-api-access-dxb6w\") pod \"a1ae9d3d-c90a-435b-babf-281b67f941c4\" (UID: \"a1ae9d3d-c90a-435b-babf-281b67f941c4\") "
Sep 18 19:13:57.688946 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:57.688910    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ae9d3d-c90a-435b-babf-281b67f941c4-kube-api-access-dxb6w" (OuterVolumeSpecName: "kube-api-access-dxb6w") pod "a1ae9d3d-c90a-435b-babf-281b67f941c4" (UID: "a1ae9d3d-c90a-435b-babf-281b67f941c4"). InnerVolumeSpecName "kube-api-access-dxb6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:57.787855 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:57.787810    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxb6w\" (UniqueName: \"kubernetes.io/projected/a1ae9d3d-c90a-435b-babf-281b67f941c4-kube-api-access-dxb6w\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:13:58.314238 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:58.314201    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1ae9d3d-c90a-435b-babf-281b67f941c4" path="/var/lib/kubelet/pods/a1ae9d3d-c90a-435b-babf-281b67f941c4/volumes"
Sep 18 19:13:58.412038 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:58.412005    2581 generic.go:356] "Generic (PLEG): container finished" podID="a1ae9d3d-c90a-435b-babf-281b67f941c4" containerID="205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b" exitCode=137
Sep 18 19:13:58.412247 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:58.412047    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7684-d725da"
Sep 18 19:13:58.412247 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:58.412114    2581 scope.go:117] "RemoveContainer" containerID="205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b"
Sep 18 19:13:58.421083 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:58.421062    2581 scope.go:117] "RemoveContainer" containerID="205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b"
Sep 18 19:13:58.421429 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:13:58.421409    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b\": container with ID starting with 205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b not found: ID does not exist" containerID="205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b"
Sep 18 19:13:58.421485 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:13:58.421439    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b"} err="failed to get container status \"205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b\": rpc error: code = NotFound desc = could not find container \"205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b\": container with ID starting with 205e67622af3b1a6de5820b5a0c4548e632e1ad73f58dd1966dee79ccbc3b82b not found: ID does not exist"
Sep 18 19:14:11.206537 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.206501    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"]
Sep 18 19:14:11.206952 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.206907    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1ae9d3d-c90a-435b-babf-281b67f941c4" containerName="test-curl-7684-d725da"
Sep 18 19:14:11.206952 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.206919    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1ae9d3d-c90a-435b-babf-281b67f941c4" containerName="test-curl-7684-d725da"
Sep 18 19:14:11.206952 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.206936    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy"
Sep 18 19:14:11.206952 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.206942    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d458cd2-a72f-4c7d-99a6-aa40698fd016" containerName="istio-proxy"
Sep 18 19:14:11.206952 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.206952    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy"
Sep 18 19:14:11.207134 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.206958    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy"
Sep 18 19:14:11.207134 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.207019    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe2a9424-cc6a-429f-a3f7-1bfcb7c37102" containerName="istio-proxy"
Sep 18 19:14:11.207134 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.207030    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1ae9d3d-c90a-435b-babf-281b67f941c4" containerName="test-curl-7684-d725da"
Sep 18 19:14:11.211577 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.211557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.215139 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.215117    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-feb80d\"/\"kube-root-ca.crt\""
Sep 18 19:14:11.216339 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.216316    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-feb80d\"/\"default-dockercfg-5f268\""
Sep 18 19:14:11.216491 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.216415    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-feb80d\"/\"openshift-service-ca.crt\""
Sep 18 19:14:11.216576 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.216557    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-feb80d\"/\"sub-test-model-feb80d-kserve-self-signed-certs\""
Sep 18 19:14:11.225850 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.225801    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"]
Sep 18 19:14:11.310919 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.310864    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-model-cache\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.311115 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.310918    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzl4h\" (UniqueName: \"kubernetes.io/projected/86557170-ac3f-40b0-a7e7-bab97015cb53-kube-api-access-nzl4h\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.311115 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.311007    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86557170-ac3f-40b0-a7e7-bab97015cb53-tls-certs\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.311115 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.311044    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-tmp-dir\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.311115 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.311063    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-home\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.311260 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.311126    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-dshm\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.412482 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.412437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-dshm\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.412740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.412519    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-model-cache\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.412740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.412537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzl4h\" (UniqueName: \"kubernetes.io/projected/86557170-ac3f-40b0-a7e7-bab97015cb53-kube-api-access-nzl4h\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.412740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.412560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86557170-ac3f-40b0-a7e7-bab97015cb53-tls-certs\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.412740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.412589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-tmp-dir\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.412740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.412605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-home\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.413036 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.413010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-model-cache\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.413085 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.413041    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-home\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.413085 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.413056    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-tmp-dir\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.415011 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.414989    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-dshm\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.415276 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.415259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86557170-ac3f-40b0-a7e7-bab97015cb53-tls-certs\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.425972 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.425931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzl4h\" (UniqueName: \"kubernetes.io/projected/86557170-ac3f-40b0-a7e7-bab97015cb53-kube-api-access-nzl4h\") pod \"sub-test-model-feb80d-kserve-788bc696c7-v9nsr\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") " pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.522202 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.522100    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:11.805885 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:11.805858    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"]
Sep 18 19:14:11.807136 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:14:11.807106    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86557170_ac3f_40b0_a7e7_bab97015cb53.slice/crio-be180cff6470600eb4f16997845ccba6ac2460ed5d3aa253af10cd6d590fed5b WatchSource:0}: Error finding container be180cff6470600eb4f16997845ccba6ac2460ed5d3aa253af10cd6d590fed5b: Status 404 returned error can't find the container with id be180cff6470600eb4f16997845ccba6ac2460ed5d3aa253af10cd6d590fed5b
Sep 18 19:14:12.495373 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:12.495339    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr" event={"ID":"86557170-ac3f-40b0-a7e7-bab97015cb53","Type":"ContainerStarted","Data":"8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b"}
Sep 18 19:14:12.495851 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:12.495382    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr" event={"ID":"86557170-ac3f-40b0-a7e7-bab97015cb53","Type":"ContainerStarted","Data":"be180cff6470600eb4f16997845ccba6ac2460ed5d3aa253af10cd6d590fed5b"}
Sep 18 19:14:21.522435 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:21.522348    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:21.522435 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:21.522409    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:21.535294 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:21.535256    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:21.556755 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:21.556691    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:14:21.556960 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:21.556877    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr" podStartSLOduration=10.556859782 podStartE2EDuration="10.556859782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:14:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:12.519127592 +0000 UTC m=+1788.738135169" watchObservedRunningTime="2026-09-18 19:14:21.556859782 +0000 UTC m=+1797.775867363"
Sep 18 19:14:29.724352 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.724315    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"]
Sep 18 19:14:29.728216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.728189    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.732432 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.732409    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-19944d\"/\"default-dockercfg-mqls5\""
Sep 18 19:14:29.732562 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.732488    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-19944d\"/\"sub-test-model-19944d-kserve-self-signed-certs\""
Sep 18 19:14:29.732679 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.732658    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-19944d\"/\"kube-root-ca.crt\""
Sep 18 19:14:29.732765 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.732693    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-19944d\"/\"openshift-service-ca.crt\""
Sep 18 19:14:29.748107 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.748077    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"]
Sep 18 19:14:29.781850 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.781804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0305ef3-7358-44e2-9f7e-6a444620c917-tls-certs\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.781850 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.781856    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-dshm\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.782100 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.781876    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-model-cache\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.782100 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.781968    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-tmp-dir\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.782100 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.782002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf8z4\" (UniqueName: \"kubernetes.io/projected/a0305ef3-7358-44e2-9f7e-6a444620c917-kube-api-access-tf8z4\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.782100 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.782025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-home\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.882939 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.882897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0305ef3-7358-44e2-9f7e-6a444620c917-tls-certs\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.882939 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.882947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-dshm\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.883238 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.882969    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-model-cache\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.883238 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.883003    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-tmp-dir\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.883238 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.883023    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf8z4\" (UniqueName: \"kubernetes.io/projected/a0305ef3-7358-44e2-9f7e-6a444620c917-kube-api-access-tf8z4\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.883238 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.883054    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-home\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.883507 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.883480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-model-cache\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.883752 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.883730    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-home\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.883840 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.883757    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-tmp-dir\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.886122 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.886094    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-dshm\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.886342 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.886320    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0305ef3-7358-44e2-9f7e-6a444620c917-tls-certs\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:29.894365 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:29.894332    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf8z4\" (UniqueName: \"kubernetes.io/projected/a0305ef3-7358-44e2-9f7e-6a444620c917-kube-api-access-tf8z4\") pod \"sub-test-model-19944d-kserve-545b74b445-kj7x7\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") " pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:30.038167 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:30.038071    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:30.348614 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:30.348586    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"]
Sep 18 19:14:30.350494 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:14:30.350468    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0305ef3_7358_44e2_9f7e_6a444620c917.slice/crio-5963b0898b0795b398296ee79f3722264bdc53d1dc2fac550105c8d2d9319ca3 WatchSource:0}: Error finding container 5963b0898b0795b398296ee79f3722264bdc53d1dc2fac550105c8d2d9319ca3: Status 404 returned error can't find the container with id 5963b0898b0795b398296ee79f3722264bdc53d1dc2fac550105c8d2d9319ca3
Sep 18 19:14:30.603120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:30.603018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7" event={"ID":"a0305ef3-7358-44e2-9f7e-6a444620c917","Type":"ContainerStarted","Data":"042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d"}
Sep 18 19:14:30.603120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:30.603075    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7" event={"ID":"a0305ef3-7358-44e2-9f7e-6a444620c917","Type":"ContainerStarted","Data":"5963b0898b0795b398296ee79f3722264bdc53d1dc2fac550105c8d2d9319ca3"}
Sep 18 19:14:30.621905 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:30.621848    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7" podStartSLOduration=1.621828788 podStartE2EDuration="1.621828788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:14:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:14:30.621157088 +0000 UTC m=+1806.840164690" watchObservedRunningTime="2026-09-18 19:14:30.621828788 +0000 UTC m=+1806.840836364"
Sep 18 19:14:40.038622 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:40.038579    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:40.039114 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:40.038645    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:40.052188 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:40.052150    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:14:40.673624 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:14:40.673587    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:15:00.190026 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:00.189983    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst"]
Sep 18 19:15:00.193717 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:00.193683    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst"
Sep 18 19:15:00.204678 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:00.204650    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:15:00.214490 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:00.214458    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst"
Sep 18 19:15:00.231664 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:00.231630    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst"]
Sep 18 19:15:00.495066 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:00.495029    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst"]
Sep 18 19:15:00.496984 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:15:00.496944    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d2a12b_9c14_4d59_a5c6_c30b05f83333.slice/crio-bd62413f6586c22bc812b68ff9134bb47511f47556907b6c9b3cb87549b757d3 WatchSource:0}: Error finding container bd62413f6586c22bc812b68ff9134bb47511f47556907b6c9b3cb87549b757d3: Status 404 returned error can't find the container with id bd62413f6586c22bc812b68ff9134bb47511f47556907b6c9b3cb87549b757d3
Sep 18 19:15:00.780919 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:00.780823    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst" event={"ID":"56d2a12b-9c14-4d59-a5c6-c30b05f83333","Type":"ContainerStarted","Data":"bd62413f6586c22bc812b68ff9134bb47511f47556907b6c9b3cb87549b757d3"}
Sep 18 19:15:03.801539 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:03.801423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst" event={"ID":"56d2a12b-9c14-4d59-a5c6-c30b05f83333","Type":"ContainerStarted","Data":"b5068f0e5bd11765e646c191a883a78ac7870f68fa828912fa2bca4eb87ff32a"}
Sep 18 19:15:03.832980 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:03.832911    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst" podStartSLOduration=1.120275701 podStartE2EDuration="3.832894587s" totalImagesPullingTime="2.712618886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:15:03.831979121 +0000 UTC m=+1840.050986699" watchObservedRunningTime="2026-09-18 19:15:03.832894587 +0000 UTC m=+1840.051902159"
Sep 18 19:15:04.808103 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:04.808071    2581 generic.go:356] "Generic (PLEG): container finished" podID="56d2a12b-9c14-4d59-a5c6-c30b05f83333" containerID="b5068f0e5bd11765e646c191a883a78ac7870f68fa828912fa2bca4eb87ff32a" exitCode=0
Sep 18 19:15:04.808505 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:04.808142    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst" event={"ID":"56d2a12b-9c14-4d59-a5c6-c30b05f83333","Type":"ContainerDied","Data":"b5068f0e5bd11765e646c191a883a78ac7870f68fa828912fa2bca4eb87ff32a"}
Sep 18 19:15:05.957413 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:05.957380    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst"
Sep 18 19:15:06.823858 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:06.823822    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst"
Sep 18 19:15:06.823858 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:06.823849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst" event={"ID":"56d2a12b-9c14-4d59-a5c6-c30b05f83333","Type":"ContainerDied","Data":"bd62413f6586c22bc812b68ff9134bb47511f47556907b6c9b3cb87549b757d3"}
Sep 18 19:15:06.824129 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:06.823919    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd62413f6586c22bc812b68ff9134bb47511f47556907b6c9b3cb87549b757d3"
Sep 18 19:15:19.575774 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:19.575731    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"]
Sep 18 19:15:20.575425 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:20.575390    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"]
Sep 18 19:15:34.595615 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:34.595563    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr" podUID="86557170-ac3f-40b0-a7e7-bab97015cb53" containerName="main" containerID="cri-o://8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b" gracePeriod=45
Sep 18 19:15:34.865477 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:34.865440    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:15:34.989117 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:34.989073    2581 generic.go:356] "Generic (PLEG): container finished" podID="86557170-ac3f-40b0-a7e7-bab97015cb53" containerID="8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b" exitCode=0
Sep 18 19:15:34.989348 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:34.989129    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"
Sep 18 19:15:34.989348 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:34.989154    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr" event={"ID":"86557170-ac3f-40b0-a7e7-bab97015cb53","Type":"ContainerDied","Data":"8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b"}
Sep 18 19:15:34.989348 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:34.989208    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr" event={"ID":"86557170-ac3f-40b0-a7e7-bab97015cb53","Type":"ContainerDied","Data":"be180cff6470600eb4f16997845ccba6ac2460ed5d3aa253af10cd6d590fed5b"}
Sep 18 19:15:34.989348 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:34.989233    2581 scope.go:117] "RemoveContainer" containerID="8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b"
Sep 18 19:15:34.998768 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:34.998742    2581 scope.go:117] "RemoveContainer" containerID="8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b"
Sep 18 19:15:34.999157 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:15:34.999129    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b\": container with ID starting with 8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b not found: ID does not exist" containerID="8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b"
Sep 18 19:15:34.999274 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:34.999165    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b"} err="failed to get container status \"8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b\": rpc error: code = NotFound desc = could not find container \"8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b\": container with ID starting with 8ed8968eac41e3f701e1afb7881c76143abedd7ee372dbefd89624330ccfa55b not found: ID does not exist"
Sep 18 19:15:35.024540 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.024503    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-home\") pod \"86557170-ac3f-40b0-a7e7-bab97015cb53\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") "
Sep 18 19:15:35.024762 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.024597    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzl4h\" (UniqueName: \"kubernetes.io/projected/86557170-ac3f-40b0-a7e7-bab97015cb53-kube-api-access-nzl4h\") pod \"86557170-ac3f-40b0-a7e7-bab97015cb53\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") "
Sep 18 19:15:35.024762 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.024628    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86557170-ac3f-40b0-a7e7-bab97015cb53-tls-certs\") pod \"86557170-ac3f-40b0-a7e7-bab97015cb53\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") "
Sep 18 19:15:35.024762 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.024654    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-dshm\") pod \"86557170-ac3f-40b0-a7e7-bab97015cb53\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") "
Sep 18 19:15:35.024762 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.024682    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-tmp-dir\") pod \"86557170-ac3f-40b0-a7e7-bab97015cb53\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") "
Sep 18 19:15:35.024991 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.024773    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-model-cache\") pod \"86557170-ac3f-40b0-a7e7-bab97015cb53\" (UID: \"86557170-ac3f-40b0-a7e7-bab97015cb53\") "
Sep 18 19:15:35.024991 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.024838    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-home" (OuterVolumeSpecName: "home") pod "86557170-ac3f-40b0-a7e7-bab97015cb53" (UID: "86557170-ac3f-40b0-a7e7-bab97015cb53"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:15:35.025096 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.025032    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "86557170-ac3f-40b0-a7e7-bab97015cb53" (UID: "86557170-ac3f-40b0-a7e7-bab97015cb53"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:15:35.025096 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.025065    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:35.025226 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.025205    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-model-cache" (OuterVolumeSpecName: "model-cache") pod "86557170-ac3f-40b0-a7e7-bab97015cb53" (UID: "86557170-ac3f-40b0-a7e7-bab97015cb53"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:15:35.027270 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.027231    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86557170-ac3f-40b0-a7e7-bab97015cb53-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "86557170-ac3f-40b0-a7e7-bab97015cb53" (UID: "86557170-ac3f-40b0-a7e7-bab97015cb53"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:15:35.027270 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.027259    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86557170-ac3f-40b0-a7e7-bab97015cb53-kube-api-access-nzl4h" (OuterVolumeSpecName: "kube-api-access-nzl4h") pod "86557170-ac3f-40b0-a7e7-bab97015cb53" (UID: "86557170-ac3f-40b0-a7e7-bab97015cb53"). InnerVolumeSpecName "kube-api-access-nzl4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:15:35.027270 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.027259    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-dshm" (OuterVolumeSpecName: "dshm") pod "86557170-ac3f-40b0-a7e7-bab97015cb53" (UID: "86557170-ac3f-40b0-a7e7-bab97015cb53"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:15:35.126476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.126377    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:35.126476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.126411    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:35.126476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.126422    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86557170-ac3f-40b0-a7e7-bab97015cb53-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:35.126476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.126435    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzl4h\" (UniqueName: \"kubernetes.io/projected/86557170-ac3f-40b0-a7e7-bab97015cb53-kube-api-access-nzl4h\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:35.126476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.126446    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86557170-ac3f-40b0-a7e7-bab97015cb53-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:35.323201 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.323162    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"]
Sep 18 19:15:35.342107 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.342065    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-feb80d/sub-test-model-feb80d-kserve-788bc696c7-v9nsr"]
Sep 18 19:15:35.595541 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.595498    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7" podUID="a0305ef3-7358-44e2-9f7e-6a444620c917" containerName="main" containerID="cri-o://042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d" gracePeriod=45
Sep 18 19:15:35.859943 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.859866    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:15:35.996124 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.996087    2581 generic.go:356] "Generic (PLEG): container finished" podID="a0305ef3-7358-44e2-9f7e-6a444620c917" containerID="042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d" exitCode=0
Sep 18 19:15:35.996345 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.996141    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"
Sep 18 19:15:35.996345 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.996174    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7" event={"ID":"a0305ef3-7358-44e2-9f7e-6a444620c917","Type":"ContainerDied","Data":"042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d"}
Sep 18 19:15:35.996345 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.996223    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7" event={"ID":"a0305ef3-7358-44e2-9f7e-6a444620c917","Type":"ContainerDied","Data":"5963b0898b0795b398296ee79f3722264bdc53d1dc2fac550105c8d2d9319ca3"}
Sep 18 19:15:35.996345 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:35.996249    2581 scope.go:117] "RemoveContainer" containerID="042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d"
Sep 18 19:15:36.005487 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.005462    2581 scope.go:117] "RemoveContainer" containerID="042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d"
Sep 18 19:15:36.005838 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:15:36.005809    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d\": container with ID starting with 042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d not found: ID does not exist" containerID="042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d"
Sep 18 19:15:36.005901 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.005850    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d"} err="failed to get container status \"042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d\": rpc error: code = NotFound desc = could not find container \"042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d\": container with ID starting with 042ba472d3aaa88de80eb5a1738d2964aec959f52eebdd330a9816014f18822d not found: ID does not exist"
Sep 18 19:15:36.035815 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.035780    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-home\") pod \"a0305ef3-7358-44e2-9f7e-6a444620c917\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") "
Sep 18 19:15:36.035815 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.035821    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-dshm\") pod \"a0305ef3-7358-44e2-9f7e-6a444620c917\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") "
Sep 18 19:15:36.036063 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.035855    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf8z4\" (UniqueName: \"kubernetes.io/projected/a0305ef3-7358-44e2-9f7e-6a444620c917-kube-api-access-tf8z4\") pod \"a0305ef3-7358-44e2-9f7e-6a444620c917\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") "
Sep 18 19:15:36.036063 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.035887    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0305ef3-7358-44e2-9f7e-6a444620c917-tls-certs\") pod \"a0305ef3-7358-44e2-9f7e-6a444620c917\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") "
Sep 18 19:15:36.036063 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.035933    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-model-cache\") pod \"a0305ef3-7358-44e2-9f7e-6a444620c917\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") "
Sep 18 19:15:36.036063 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.036013    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-tmp-dir\") pod \"a0305ef3-7358-44e2-9f7e-6a444620c917\" (UID: \"a0305ef3-7358-44e2-9f7e-6a444620c917\") "
Sep 18 19:15:36.036267 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.036092    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-home" (OuterVolumeSpecName: "home") pod "a0305ef3-7358-44e2-9f7e-6a444620c917" (UID: "a0305ef3-7358-44e2-9f7e-6a444620c917"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:15:36.036319 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.036269    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-model-cache" (OuterVolumeSpecName: "model-cache") pod "a0305ef3-7358-44e2-9f7e-6a444620c917" (UID: "a0305ef3-7358-44e2-9f7e-6a444620c917"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:15:36.036374 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.036310    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a0305ef3-7358-44e2-9f7e-6a444620c917" (UID: "a0305ef3-7358-44e2-9f7e-6a444620c917"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:15:36.036429 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.036398    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:36.036429 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.036417    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:36.038472 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.038436    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0305ef3-7358-44e2-9f7e-6a444620c917-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a0305ef3-7358-44e2-9f7e-6a444620c917" (UID: "a0305ef3-7358-44e2-9f7e-6a444620c917"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:15:36.038602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.038498    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-dshm" (OuterVolumeSpecName: "dshm") pod "a0305ef3-7358-44e2-9f7e-6a444620c917" (UID: "a0305ef3-7358-44e2-9f7e-6a444620c917"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:15:36.038602 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.038522    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0305ef3-7358-44e2-9f7e-6a444620c917-kube-api-access-tf8z4" (OuterVolumeSpecName: "kube-api-access-tf8z4") pod "a0305ef3-7358-44e2-9f7e-6a444620c917" (UID: "a0305ef3-7358-44e2-9f7e-6a444620c917"). InnerVolumeSpecName "kube-api-access-tf8z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:15:36.137397 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.137296    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:36.137397 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.137333    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0305ef3-7358-44e2-9f7e-6a444620c917-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:36.137397 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.137342    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf8z4\" (UniqueName: \"kubernetes.io/projected/a0305ef3-7358-44e2-9f7e-6a444620c917-kube-api-access-tf8z4\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:36.137397 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.137352    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0305ef3-7358-44e2-9f7e-6a444620c917-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:15:36.314654 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.314619    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86557170-ac3f-40b0-a7e7-bab97015cb53" path="/var/lib/kubelet/pods/86557170-ac3f-40b0-a7e7-bab97015cb53/volumes"
Sep 18 19:15:36.330371 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.330329    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"]
Sep 18 19:15:36.339658 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:36.339622    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/sub-test-model-19944d-kserve-545b74b445-kj7x7"]
Sep 18 19:15:38.313446 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:38.313402    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0305ef3-7358-44e2-9f7e-6a444620c917" path="/var/lib/kubelet/pods/a0305ef3-7358-44e2-9f7e-6a444620c917/volumes"
Sep 18 19:15:44.530202 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.530151    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"]
Sep 18 19:15:44.530754 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.530694    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86557170-ac3f-40b0-a7e7-bab97015cb53" containerName="main"
Sep 18 19:15:44.530754 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.530737    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="86557170-ac3f-40b0-a7e7-bab97015cb53" containerName="main"
Sep 18 19:15:44.530909 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.530774    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0305ef3-7358-44e2-9f7e-6a444620c917" containerName="main"
Sep 18 19:15:44.530909 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.530783    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0305ef3-7358-44e2-9f7e-6a444620c917" containerName="main"
Sep 18 19:15:44.530909 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.530799    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56d2a12b-9c14-4d59-a5c6-c30b05f83333" containerName="cleanup"
Sep 18 19:15:44.530909 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.530807    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56d2a12b-9c14-4d59-a5c6-c30b05f83333" containerName="cleanup"
Sep 18 19:15:44.531099 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.530934    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0305ef3-7358-44e2-9f7e-6a444620c917" containerName="main"
Sep 18 19:15:44.531099 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.530948    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="56d2a12b-9c14-4d59-a5c6-c30b05f83333" containerName="cleanup"
Sep 18 19:15:44.531099 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.530958    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="86557170-ac3f-40b0-a7e7-bab97015cb53" containerName="main"
Sep 18 19:15:44.535400 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.535369    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.546164 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.546133    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-19944d\"/\"rate-test-model-19944d-kserve-self-signed-certs\""
Sep 18 19:15:44.547191 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.547173    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-19944d\"/\"kube-root-ca.crt\""
Sep 18 19:15:44.547979 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.547960    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-19944d\"/\"openshift-service-ca.crt\""
Sep 18 19:15:44.549254 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.549236    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-19944d\"/\"default-dockercfg-mqls5\""
Sep 18 19:15:44.562747 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.562692    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"]
Sep 18 19:15:44.611905 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.611857    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-model-cache\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.611905 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.611903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-dshm\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.612134 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.611947    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsz9q\" (UniqueName: \"kubernetes.io/projected/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-kube-api-access-rsz9q\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.612134 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.612038    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tls-certs\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.612134 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.612083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tmp-dir\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.612233 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.612140    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-home\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.712832 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.712790    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tls-certs\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.713051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.712856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tmp-dir\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.713051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.712903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-home\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.713051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.712942    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-model-cache\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.713051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.712976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-dshm\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.713051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.713009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsz9q\" (UniqueName: \"kubernetes.io/projected/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-kube-api-access-rsz9q\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.713378 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.713340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tmp-dir\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.713449 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.713423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-model-cache\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.713550 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.713522    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-home\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.715557 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.715523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-dshm\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.716010 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.715989    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tls-certs\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.728742 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.728684    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsz9q\" (UniqueName: \"kubernetes.io/projected/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-kube-api-access-rsz9q\") pod \"rate-test-model-19944d-kserve-d4868c9b7-7jpsg\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") " pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:44.845081 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:44.844982    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:45.204450 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:15:45.204417    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfa5e933_0a2f_44b3_9997_d1d4172b6ef9.slice/crio-e91dbee22e6c65f55d61bde9dd08d16fc06af2788842e30ae26ebd1f487fb77c WatchSource:0}: Error finding container e91dbee22e6c65f55d61bde9dd08d16fc06af2788842e30ae26ebd1f487fb77c: Status 404 returned error can't find the container with id e91dbee22e6c65f55d61bde9dd08d16fc06af2788842e30ae26ebd1f487fb77c
Sep 18 19:15:45.205304 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:45.205271    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"]
Sep 18 19:15:46.057047 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:46.057011    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg" event={"ID":"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9","Type":"ContainerStarted","Data":"6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e"}
Sep 18 19:15:46.057047 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:46.057057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg" event={"ID":"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9","Type":"ContainerStarted","Data":"e91dbee22e6c65f55d61bde9dd08d16fc06af2788842e30ae26ebd1f487fb77c"}
Sep 18 19:15:54.845149 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:54.845112    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:54.845646 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:54.845204    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:54.858612 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:54.858582    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:15:54.913811 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:54.913737    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg" podStartSLOduration=10.913717669 podStartE2EDuration="10.913717669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:15:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:15:46.109230129 +0000 UTC m=+1882.328237706" watchObservedRunningTime="2026-09-18 19:15:54.913717669 +0000 UTC m=+1891.132725246"
Sep 18 19:15:55.119512 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:15:55.119424    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:16:31.285882 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:31.285834    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"]
Sep 18 19:16:32.146773 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.146731    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"]
Sep 18 19:16:32.151579 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.151558    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.162971 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.162949    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-f8cdfc-openshift-default-dockercfg-9vstm\""
Sep 18 19:16:32.170968 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.170937    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"]
Sep 18 19:16:32.260030 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.259988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-token\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.260030 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.260033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a74eac0a-7c51-4404-a978-2416ae665fbf-istiod-ca-cert\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.260278 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.260070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-data\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.260278 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.260114    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-credential-socket\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.260278 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.260142    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-envoy\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.260278 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.260163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-podinfo\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.260278 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.260184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c57qp\" (UniqueName: \"kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-kube-api-access-c57qp\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.260278 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.260215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-socket\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.260278 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.260247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-certs\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.360736 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.360664    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-data\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.360768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-credential-socket\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.360801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-envoy\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.360822    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-podinfo\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.360846    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c57qp\" (UniqueName: \"kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-kube-api-access-c57qp\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.360874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-socket\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.360888    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-certs\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.360924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-token\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.360953    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a74eac0a-7c51-4404-a978-2416ae665fbf-istiod-ca-cert\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.361165    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-data\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.361187    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-credential-socket\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361665 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.361332    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-socket\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361665 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.361488    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-certs\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.361665 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.361589    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a74eac0a-7c51-4404-a978-2416ae665fbf-istiod-ca-cert\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.363447 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.363423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-envoy\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.363631 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.363610    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-podinfo\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.373051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.373016    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c57qp\" (UniqueName: \"kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-kube-api-access-c57qp\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.373214 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.373196    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-token\") pod \"e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") " pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.461645 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.461596    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:32.770377 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.770348    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"]
Sep 18 19:16:32.770847 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:16:32.770827    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda74eac0a_7c51_4404_a978_2416ae665fbf.slice/crio-87ffb83e87fcbfc5cbce03352203c50939fc3a321b01db74af28c2f5fd7a994d WatchSource:0}: Error finding container 87ffb83e87fcbfc5cbce03352203c50939fc3a321b01db74af28c2f5fd7a994d: Status 404 returned error can't find the container with id 87ffb83e87fcbfc5cbce03352203c50939fc3a321b01db74af28c2f5fd7a994d
Sep 18 19:16:32.773048 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.773017    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:16:32.773129 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.773086    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:16:32.773129 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:32.773114    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:16:33.329395 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:33.329349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" event={"ID":"a74eac0a-7c51-4404-a978-2416ae665fbf","Type":"ContainerStarted","Data":"18de322fd7586879e864a3c696daad7fc5bf938bcd11302707f3197731df33da"}
Sep 18 19:16:33.329622 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:33.329402    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" event={"ID":"a74eac0a-7c51-4404-a978-2416ae665fbf","Type":"ContainerStarted","Data":"87ffb83e87fcbfc5cbce03352203c50939fc3a321b01db74af28c2f5fd7a994d"}
Sep 18 19:16:33.462461 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:33.462412    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:33.467992 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:33.467960    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:33.508909 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:33.508847    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" podStartSLOduration=1.508821242 podStartE2EDuration="1.508821242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:33.36552885 +0000 UTC m=+1929.584536427" watchObservedRunningTime="2026-09-18 19:16:33.508821242 +0000 UTC m=+1929.727828819"
Sep 18 19:16:34.336772 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:34.336730    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:34.337847 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:34.337815    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:46.306914 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.306852    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg" podUID="cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" containerName="main" containerID="cri-o://6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e" gracePeriod=45
Sep 18 19:16:46.565823 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.565741    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:16:46.707904 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.707864    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-model-cache\") pod \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") "
Sep 18 19:16:46.708094 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.707945    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-home\") pod \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") "
Sep 18 19:16:46.708094 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.708023    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsz9q\" (UniqueName: \"kubernetes.io/projected/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-kube-api-access-rsz9q\") pod \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") "
Sep 18 19:16:46.708094 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.708075    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-dshm\") pod \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") "
Sep 18 19:16:46.708094 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.708094    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tls-certs\") pod \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") "
Sep 18 19:16:46.708319 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.708117    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tmp-dir\") pod \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\" (UID: \"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9\") "
Sep 18 19:16:46.708319 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.708225    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-home" (OuterVolumeSpecName: "home") pod "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" (UID: "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:16:46.708319 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.708241    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-model-cache" (OuterVolumeSpecName: "model-cache") pod "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" (UID: "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:16:46.708477 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.708395    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:16:46.708477 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.708441    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:16:46.708549 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.708489    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" (UID: "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:16:46.710375 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.710341    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-dshm" (OuterVolumeSpecName: "dshm") pod "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" (UID: "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:16:46.710503 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.710385    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-kube-api-access-rsz9q" (OuterVolumeSpecName: "kube-api-access-rsz9q") pod "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" (UID: "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9"). InnerVolumeSpecName "kube-api-access-rsz9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:16:46.710503 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.710459    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" (UID: "cfa5e933-0a2f-44b3-9997-d1d4172b6ef9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:16:46.810045 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.809981    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsz9q\" (UniqueName: \"kubernetes.io/projected/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-kube-api-access-rsz9q\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:16:46.810045 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.810020    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:16:46.810045 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.810031    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:16:46.810045 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:46.810058    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:16:47.419317 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:47.419281    2581 generic.go:356] "Generic (PLEG): container finished" podID="cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" containerID="6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e" exitCode=0
Sep 18 19:16:47.419858 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:47.419326    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"
Sep 18 19:16:47.419858 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:47.419375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg" event={"ID":"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9","Type":"ContainerDied","Data":"6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e"}
Sep 18 19:16:47.419858 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:47.419414    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg" event={"ID":"cfa5e933-0a2f-44b3-9997-d1d4172b6ef9","Type":"ContainerDied","Data":"e91dbee22e6c65f55d61bde9dd08d16fc06af2788842e30ae26ebd1f487fb77c"}
Sep 18 19:16:47.419858 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:47.419435    2581 scope.go:117] "RemoveContainer" containerID="6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e"
Sep 18 19:16:47.434907 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:47.434780    2581 scope.go:117] "RemoveContainer" containerID="6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e"
Sep 18 19:16:47.435294 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:16:47.435235    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e\": container with ID starting with 6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e not found: ID does not exist" containerID="6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e"
Sep 18 19:16:47.435916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:47.435284    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e"} err="failed to get container status \"6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e\": rpc error: code = NotFound desc = could not find container \"6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e\": container with ID starting with 6c6cfbffd85618c272770a34d696c291ce8368341d9dd3a1ba115864c99bc49e not found: ID does not exist"
Sep 18 19:16:47.469756 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:47.469718    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"]
Sep 18 19:16:47.474609 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:47.474576    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-19944d/rate-test-model-19944d-kserve-d4868c9b7-7jpsg"]
Sep 18 19:16:48.314004 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:48.313972    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" path="/var/lib/kubelet/pods/cfa5e933-0a2f-44b3-9997-d1d4172b6ef9/volumes"
Sep 18 19:16:56.484609 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:56.484572    2581 generic.go:356] "Generic (PLEG): container finished" podID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerID="18de322fd7586879e864a3c696daad7fc5bf938bcd11302707f3197731df33da" exitCode=0
Sep 18 19:16:56.485062 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:56.484645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" event={"ID":"a74eac0a-7c51-4404-a978-2416ae665fbf","Type":"ContainerDied","Data":"18de322fd7586879e864a3c696daad7fc5bf938bcd11302707f3197731df33da"}
Sep 18 19:16:56.485106 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:56.485070    2581 scope.go:117] "RemoveContainer" containerID="18de322fd7586879e864a3c696daad7fc5bf938bcd11302707f3197731df33da"
Sep 18 19:16:56.486145 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:56.486118    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:16:56.486215 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:56.486184    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:16:56.486253 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:56.486220    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:16:57.493102 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:57.493063    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" event={"ID":"a74eac0a-7c51-4404-a978-2416ae665fbf","Type":"ContainerStarted","Data":"fccccd04e83c0381a5338df646fabe91217377585e6c94b1dec13f7002a7835a"}
Sep 18 19:16:58.462487 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:58.462441    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:16:58.462696 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:58.462674    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 18 19:16:58.462780 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:58.462751    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 18 19:16:59.463047 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:59.463009    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 18 19:16:59.463489 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:16:59.463079    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 18 19:17:00.462760 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:00.462689    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 18 19:17:00.462960 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:00.462793    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 18 19:17:01.462249 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:01.462206    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 18 19:17:01.462666 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:01.462266    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 18 19:17:02.463017 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:02.462970    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 18 19:17:02.463470 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:02.463038    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 18 19:17:03.467474 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:03.467439    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:17:03.467898 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:03.467834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:17:03.468488 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:03.468472    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:17:07.224540 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.224500    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"]
Sep 18 19:17:07.225029 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.225014    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" containerName="main"
Sep 18 19:17:07.225079 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.225032    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" containerName="main"
Sep 18 19:17:07.225166 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.225152    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfa5e933-0a2f-44b3-9997-d1d4172b6ef9" containerName="main"
Sep 18 19:17:07.228758 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.228724    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.239438 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:07.239404    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.239583 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:07.239504    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-ipp-b-8f9642-openshift-default-dockercfg-2lggh\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-8f9642-openshift-default-dockercfg-2lggh\"" type="*v1.Secret"
Sep 18 19:17:07.301251 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.301212    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"]
Sep 18 19:17:07.314520 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.314474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istiod-ca-cert\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.314760 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.314549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-token\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.314760 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.314578    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-podinfo\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.314760 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.314620    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-envoy\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.314760 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.314672    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-credential-socket\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.314760 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.314726    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-certs\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.314760 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.314758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-data\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.315058 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.314787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvpmr\" (UniqueName: \"kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-kube-api-access-vvpmr\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.315058 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.314850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-socket\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416147 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416106    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istiod-ca-cert\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416147 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416155    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-token\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416411 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416173    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-podinfo\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416411 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-envoy\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416411 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416237    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-credential-socket\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416411 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-certs\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416411 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416274    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-data\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416411 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvpmr\" (UniqueName: \"kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-kube-api-access-vvpmr\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416411 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-socket\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416904 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416800    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-credential-socket\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416904 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-certs\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.416904 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416870    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-socket\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.417058 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.416965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-data\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.417254 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.417228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istiod-ca-cert\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.419133 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.419105    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-podinfo\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.419448 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.419426    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-envoy\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.432653 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.432621    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvpmr\" (UniqueName: \"kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-kube-api-access-vvpmr\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:07.449992 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:07.449961    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-token\") pod \"e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") " pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:08.304558 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:08.304523    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-8f9642-openshift-default-dockercfg-2lggh\""
Sep 18 19:17:08.311125 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:08.311099    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:08.633143 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:08.633109    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"]
Sep 18 19:17:08.635191 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:17:08.635153    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7e0084c_1782_41b0_8a4b_09337b0b20e8.slice/crio-02c8a156310aaf06ff6a64767d5044e867a977655bb56b97495146b97cf70d16 WatchSource:0}: Error finding container 02c8a156310aaf06ff6a64767d5044e867a977655bb56b97495146b97cf70d16: Status 404 returned error can't find the container with id 02c8a156310aaf06ff6a64767d5044e867a977655bb56b97495146b97cf70d16
Sep 18 19:17:08.637887 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:08.637832    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:17:08.637991 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:08.637944    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:17:08.638051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:08.637989    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:17:09.563968 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:09.563932    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" event={"ID":"c7e0084c-1782-41b0-8a4b-09337b0b20e8","Type":"ContainerStarted","Data":"0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a"}
Sep 18 19:17:09.564406 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:09.563987    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" event={"ID":"c7e0084c-1782-41b0-8a4b-09337b0b20e8","Type":"ContainerStarted","Data":"02c8a156310aaf06ff6a64767d5044e867a977655bb56b97495146b97cf70d16"}
Sep 18 19:17:09.601437 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:09.601374    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" podStartSLOduration=2.601355892 podStartE2EDuration="2.601355892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:17:09.597885248 +0000 UTC m=+1965.816892822" watchObservedRunningTime="2026-09-18 19:17:09.601355892 +0000 UTC m=+1965.820363470"
Sep 18 19:17:10.315686 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:10.315658    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:10.317720 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:10.317674    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:10.570566 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:10.570481    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:10.571641 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:10.571619    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:17.899749 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:17.899619    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"]
Sep 18 19:17:17.903429 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:17.903406    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:17.908484 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:17.908456    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-8f9642-serving-cert\""
Sep 18 19:17:17.938172 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:17.938118    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"]
Sep 18 19:17:18.018666 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.018625    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptq2q\" (UniqueName: \"kubernetes.io/projected/cb52db18-5ddf-45f6-80e6-f8c7004eca81-kube-api-access-ptq2q\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.018893 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.018687    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.018893 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.018764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-metrics-certs\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.018893 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.018838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb52db18-5ddf-45f6-80e6-f8c7004eca81-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.119681 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.119634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-metrics-certs\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.119910 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.119757    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb52db18-5ddf-45f6-80e6-f8c7004eca81-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.119910 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.119821    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptq2q\" (UniqueName: \"kubernetes.io/projected/cb52db18-5ddf-45f6-80e6-f8c7004eca81-kube-api-access-ptq2q\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.119910 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.119876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.120041 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:18.119980    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-serving-cert: secret "maas-api-e2e-ipp-b-8f9642-serving-cert" not found
Sep 18 19:17:18.120082 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:18.120062    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-maas-api-tls podName:cb52db18-5ddf-45f6-80e6-f8c7004eca81 nodeName:}" failed. No retries permitted until 2026-09-18 19:17:18.620041546 +0000 UTC m=+1974.839049102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-maas-api-tls") pod "maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9" (UID: "cb52db18-5ddf-45f6-80e6-f8c7004eca81") : secret "maas-api-e2e-ipp-b-8f9642-serving-cert" not found
Sep 18 19:17:18.120522 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.120501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb52db18-5ddf-45f6-80e6-f8c7004eca81-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.122413 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.122388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-metrics-certs\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.135133 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.135096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptq2q\" (UniqueName: \"kubernetes.io/projected/cb52db18-5ddf-45f6-80e6-f8c7004eca81-kube-api-access-ptq2q\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.624774 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.624694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.627552 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.627521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:18.817203 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:18.817140    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:19.144642 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:19.144611    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"]
Sep 18 19:17:19.145963 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:17:19.145942    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb52db18_5ddf_45f6_80e6_f8c7004eca81.slice/crio-44635a6d91105392f873fd431b83f94ea0cd177cfdbc94a6c8a7d1b09022b389 WatchSource:0}: Error finding container 44635a6d91105392f873fd431b83f94ea0cd177cfdbc94a6c8a7d1b09022b389: Status 404 returned error can't find the container with id 44635a6d91105392f873fd431b83f94ea0cd177cfdbc94a6c8a7d1b09022b389
Sep 18 19:17:19.147399 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:19.147379    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:17:19.641495 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:19.641331    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9" event={"ID":"cb52db18-5ddf-45f6-80e6-f8c7004eca81","Type":"ContainerStarted","Data":"f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903"}
Sep 18 19:17:19.641495 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:19.641402    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9" event={"ID":"cb52db18-5ddf-45f6-80e6-f8c7004eca81","Type":"ContainerStarted","Data":"44635a6d91105392f873fd431b83f94ea0cd177cfdbc94a6c8a7d1b09022b389"}
Sep 18 19:17:19.641816 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:19.641634    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:19.706614 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:19.706555    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9" podStartSLOduration=2.310894112 podStartE2EDuration="2.706539168s" totalImagesPullingTime="395.645056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:17:19.706389688 +0000 UTC m=+1975.925397277" watchObservedRunningTime="2026-09-18 19:17:19.706539168 +0000 UTC m=+1975.925546746"
Sep 18 19:17:25.655877 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:25.655847    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:17:34.737430 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:34.737391    2581 generic.go:356] "Generic (PLEG): container finished" podID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerID="0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a" exitCode=0
Sep 18 19:17:34.737962 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:34.737442    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" event={"ID":"c7e0084c-1782-41b0-8a4b-09337b0b20e8","Type":"ContainerDied","Data":"0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a"}
Sep 18 19:17:34.737962 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:34.737842    2581 scope.go:117] "RemoveContainer" containerID="0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a"
Sep 18 19:17:34.738937 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:34.738908    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:17:34.739029 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:34.738976    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:17:34.739029 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:34.739008    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 19:17:35.754185 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:35.754144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" event={"ID":"c7e0084c-1782-41b0-8a4b-09337b0b20e8","Type":"ContainerStarted","Data":"942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f"}
Sep 18 19:17:36.315913 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:36.315872    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 19:17:36.316236 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:36.315946    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 19:17:36.318845 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:36.318821    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:37.312607 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:37.312568    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 19:17:37.313040 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:37.312631    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 19:17:38.312209 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:38.312178    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 19:17:38.312388 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:38.312232    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 19:17:39.312137 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:39.312103    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 19:17:39.312560 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:39.312159    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 19:17:40.312163 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:40.312123    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 19:17:40.312592 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:40.312195    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 19:17:40.570923 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:40.570817    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:41.312267 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:41.312228    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 19:17:41.312859 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:41.312321    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 19:17:42.316221 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:42.316180    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:42.317227 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:42.317203    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:17:47.441481 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:47.441447    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwd2p_c9c0d59a-eab3-4afb-ab9a-8ee23b012266/payload-processing/0.log"
Sep 18 19:17:53.724860 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.724821    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"]
Sep 18 19:17:53.730024 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.730000    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.734474 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:53.734434    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"ipp-route-f8cdfc-kserve-self-signed-certs\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-ipp-a-f8cdfc\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-ipp-a-f8cdfc\"/\"ipp-route-f8cdfc-kserve-self-signed-certs\"" type="*v1.Secret"
Sep 18 19:17:53.734474 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:53.734453    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-xlfs7\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-ipp-a-f8cdfc\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-ipp-a-f8cdfc\"/\"default-dockercfg-xlfs7\"" type="*v1.Secret"
Sep 18 19:17:53.734690 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:53.734471    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-ipp-a-f8cdfc\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-ipp-a-f8cdfc\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 18 19:17:53.734690 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:53.734666    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-ipp-a-f8cdfc\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-ipp-a-f8cdfc\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 18 19:17:53.734950 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:53.734922    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" is forbidden: User \"system:node:ip-10-0-136-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"ai-tenant-e2e-ipp-a-f8cdfc\": no relationship found between node 'ip-10-0-136-52.ec2.internal' and this object" podUID="0d0eb56a-f9ff-420e-ae37-7930e1412215" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.772858 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.772824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d0eb56a-f9ff-420e-ae37-7930e1412215-tls-certs\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.773069 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.772887    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qrtq\" (UniqueName: \"kubernetes.io/projected/0d0eb56a-f9ff-420e-ae37-7930e1412215-kube-api-access-4qrtq\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.773069 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.772914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-home\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.773069 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.772950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-dshm\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.773069 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.773003    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-tmp-dir\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.773069 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.773063    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-model-cache\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.773375 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.773351    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"]
Sep 18 19:17:53.874368 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.874332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-home\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.874368 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.874364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-dshm\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.874654 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.874389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-tmp-dir\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.874654 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.874439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-model-cache\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.874654 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.874503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d0eb56a-f9ff-420e-ae37-7930e1412215-tls-certs\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.874654 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.874533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qrtq\" (UniqueName: \"kubernetes.io/projected/0d0eb56a-f9ff-420e-ae37-7930e1412215-kube-api-access-4qrtq\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.874915 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.874834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-home\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.874915 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.874877    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-tmp-dir\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.874987 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.874971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-model-cache\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:53.876954 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:53.876931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-dshm\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:54.874778 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:54.874744    2581 secret.go:189] Couldn't get secret ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-self-signed-certs: failed to sync secret cache: timed out waiting for the condition
Sep 18 19:17:54.875241 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:54.874843    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d0eb56a-f9ff-420e-ae37-7930e1412215-tls-certs podName:0d0eb56a-f9ff-420e-ae37-7930e1412215 nodeName:}" failed. No retries permitted until 2026-09-18 19:17:55.374821911 +0000 UTC m=+2011.593829467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0d0eb56a-f9ff-420e-ae37-7930e1412215-tls-certs") pod "ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z" (UID: "0d0eb56a-f9ff-420e-ae37-7930e1412215") : failed to sync secret cache: timed out waiting for the condition
Sep 18 19:17:54.914697 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:54.914648    2581 projected.go:291] Couldn't get configMap ai-tenant-e2e-ipp-a-f8cdfc/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Sep 18 19:17:55.000513 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:55.000478    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-f8cdfc\"/\"openshift-service-ca.crt\""
Sep 18 19:17:55.005228 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:55.005202    2581 projected.go:196] Error preparing data for projected volume kube-api-access-4qrtq for pod ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z: failed to sync configmap cache: timed out waiting for the condition
Sep 18 19:17:55.005397 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:17:55.005288    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d0eb56a-f9ff-420e-ae37-7930e1412215-kube-api-access-4qrtq podName:0d0eb56a-f9ff-420e-ae37-7930e1412215 nodeName:}" failed. No retries permitted until 2026-09-18 19:17:55.505271724 +0000 UTC m=+2011.724279279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4qrtq" (UniqueName: "kubernetes.io/projected/0d0eb56a-f9ff-420e-ae37-7930e1412215-kube-api-access-4qrtq") pod "ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z" (UID: "0d0eb56a-f9ff-420e-ae37-7930e1412215") : failed to sync configmap cache: timed out waiting for the condition
Sep 18 19:17:55.112831 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:55.112794    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-f8cdfc\"/\"default-dockercfg-xlfs7\""
Sep 18 19:17:55.246546 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:55.246514    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-f8cdfc\"/\"ipp-route-f8cdfc-kserve-self-signed-certs\""
Sep 18 19:17:55.334497 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:55.334458    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-f8cdfc\"/\"kube-root-ca.crt\""
Sep 18 19:17:55.388803 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:55.388767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d0eb56a-f9ff-420e-ae37-7930e1412215-tls-certs\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:55.391507 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:55.391480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d0eb56a-f9ff-420e-ae37-7930e1412215-tls-certs\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:55.591660 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:55.591551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qrtq\" (UniqueName: \"kubernetes.io/projected/0d0eb56a-f9ff-420e-ae37-7930e1412215-kube-api-access-4qrtq\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:55.594355 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:55.594328    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qrtq\" (UniqueName: \"kubernetes.io/projected/0d0eb56a-f9ff-420e-ae37-7930e1412215-kube-api-access-4qrtq\") pod \"ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") " pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:55.840006 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:55.839951    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:17:56.137036 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:56.137005    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"]
Sep 18 19:17:56.138262 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:17:56.138237    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d0eb56a_f9ff_420e_ae37_7930e1412215.slice/crio-c5343a02b7bf781bd30ba5d66b3e0d1159938cbeb74ffc3467984a63b99cae30 WatchSource:0}: Error finding container c5343a02b7bf781bd30ba5d66b3e0d1159938cbeb74ffc3467984a63b99cae30: Status 404 returned error can't find the container with id c5343a02b7bf781bd30ba5d66b3e0d1159938cbeb74ffc3467984a63b99cae30
Sep 18 19:17:56.874122 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:56.874074    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z" event={"ID":"0d0eb56a-f9ff-420e-ae37-7930e1412215","Type":"ContainerStarted","Data":"9dfca032ee446fdb4cfbc02c5b665d115d0fd6494d0f1e7ca1885ab2f64a01ad"}
Sep 18 19:17:56.874122 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:56.874129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z" event={"ID":"0d0eb56a-f9ff-420e-ae37-7930e1412215","Type":"ContainerStarted","Data":"c5343a02b7bf781bd30ba5d66b3e0d1159938cbeb74ffc3467984a63b99cae30"}
Sep 18 19:17:56.904640 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:17:56.904580    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z" podStartSLOduration=3.904564859 podStartE2EDuration="3.904564859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:17:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:17:56.900575806 +0000 UTC m=+2013.119583395" watchObservedRunningTime="2026-09-18 19:17:56.904564859 +0000 UTC m=+2013.123572437"
Sep 18 19:18:05.840338 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:05.840295    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:18:05.840338 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:05.840346    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:18:05.853087 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:05.853059    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:18:05.938882 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:05.938847    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:18:29.351216 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:29.351186    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwd2p_c9c0d59a-eab3-4afb-ab9a-8ee23b012266/payload-processing/0.log"
Sep 18 19:18:40.534626 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.534583    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"]
Sep 18 19:18:40.538746 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.538696    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.542515 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.542485    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-790775-serving-cert\""
Sep 18 19:18:40.558308 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.558270    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"]
Sep 18 19:18:40.632638 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.632596    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpfl7\" (UniqueName: \"kubernetes.io/projected/c03e1f5a-4afc-4d05-8f22-48023663fb31-kube-api-access-lpfl7\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.632638 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.632646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.632950 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.632676    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.632950 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.632818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c03e1f5a-4afc-4d05-8f22-48023663fb31-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.733923 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.733881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpfl7\" (UniqueName: \"kubernetes.io/projected/c03e1f5a-4afc-4d05-8f22-48023663fb31-kube-api-access-lpfl7\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.733923 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.733929    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.734158 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.733970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.734158 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.734030    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c03e1f5a-4afc-4d05-8f22-48023663fb31-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.734158 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:18:40.734123    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-serving-cert: secret "maas-api-e2e-ipp-cleanup-790775-serving-cert" not found
Sep 18 19:18:40.734270 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:18:40.734197    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-maas-api-tls podName:c03e1f5a-4afc-4d05-8f22-48023663fb31 nodeName:}" failed. No retries permitted until 2026-09-18 19:18:41.23417919 +0000 UTC m=+2057.453186748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v" (UID: "c03e1f5a-4afc-4d05-8f22-48023663fb31") : secret "maas-api-e2e-ipp-cleanup-790775-serving-cert" not found
Sep 18 19:18:40.734766 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.734746    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c03e1f5a-4afc-4d05-8f22-48023663fb31-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.736648 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.736620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:40.762239 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:40.762188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpfl7\" (UniqueName: \"kubernetes.io/projected/c03e1f5a-4afc-4d05-8f22-48023663fb31-kube-api-access-lpfl7\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:41.239546 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:41.239504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:41.242290 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:41.242262    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:41.450099 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:41.450048    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:41.758479 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:41.758400    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"]
Sep 18 19:18:41.759153 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:18:41.759112    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc03e1f5a_4afc_4d05_8f22_48023663fb31.slice/crio-cd7770f67117418e5a44bf4e460e3e21eec38528a004b4d2403829a77fd25386 WatchSource:0}: Error finding container cd7770f67117418e5a44bf4e460e3e21eec38528a004b4d2403829a77fd25386: Status 404 returned error can't find the container with id cd7770f67117418e5a44bf4e460e3e21eec38528a004b4d2403829a77fd25386
Sep 18 19:18:42.164401 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:42.164339    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v" event={"ID":"c03e1f5a-4afc-4d05-8f22-48023663fb31","Type":"ContainerStarted","Data":"cd7770f67117418e5a44bf4e460e3e21eec38528a004b4d2403829a77fd25386"}
Sep 18 19:18:43.174257 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:43.174212    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v" event={"ID":"c03e1f5a-4afc-4d05-8f22-48023663fb31","Type":"ContainerStarted","Data":"8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0"}
Sep 18 19:18:43.174696 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:43.174462    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:18:43.219675 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:43.219600    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v" podStartSLOduration=2.594711043 podStartE2EDuration="3.219580944s" totalImagesPullingTime="624.869901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:18:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:18:43.218036852 +0000 UTC m=+2059.437044430" watchObservedRunningTime="2026-09-18 19:18:43.219580944 +0000 UTC m=+2059.438588555"
Sep 18 19:18:49.186001 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:18:49.185967    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:19:20.798771 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:20.798729    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"]
Sep 18 19:19:20.799293 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:20.798951    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v" podUID="c03e1f5a-4afc-4d05-8f22-48023663fb31" containerName="maas-api" containerID="cri-o://8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0" gracePeriod=30
Sep 18 19:19:20.849729 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:19:20.849662    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc03e1f5a_4afc_4d05_8f22_48023663fb31.slice/crio-conmon-8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0.scope\": RecentStats: unable to find data in memory cache]"
Sep 18 19:19:21.067463 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.067431    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:19:21.130476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.129697    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c03e1f5a-4afc-4d05-8f22-48023663fb31-openshift-service-ca\") pod \"c03e1f5a-4afc-4d05-8f22-48023663fb31\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") "
Sep 18 19:19:21.130476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.129801    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpfl7\" (UniqueName: \"kubernetes.io/projected/c03e1f5a-4afc-4d05-8f22-48023663fb31-kube-api-access-lpfl7\") pod \"c03e1f5a-4afc-4d05-8f22-48023663fb31\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") "
Sep 18 19:19:21.130476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.129918    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-maas-api-tls\") pod \"c03e1f5a-4afc-4d05-8f22-48023663fb31\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") "
Sep 18 19:19:21.130476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.129965    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-metrics-certs\") pod \"c03e1f5a-4afc-4d05-8f22-48023663fb31\" (UID: \"c03e1f5a-4afc-4d05-8f22-48023663fb31\") "
Sep 18 19:19:21.130982 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.130947    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c03e1f5a-4afc-4d05-8f22-48023663fb31-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c03e1f5a-4afc-4d05-8f22-48023663fb31" (UID: "c03e1f5a-4afc-4d05-8f22-48023663fb31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:19:21.133418 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.133360    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c03e1f5a-4afc-4d05-8f22-48023663fb31" (UID: "c03e1f5a-4afc-4d05-8f22-48023663fb31"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:19:21.133635 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.133560    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c03e1f5a-4afc-4d05-8f22-48023663fb31-kube-api-access-lpfl7" (OuterVolumeSpecName: "kube-api-access-lpfl7") pod "c03e1f5a-4afc-4d05-8f22-48023663fb31" (UID: "c03e1f5a-4afc-4d05-8f22-48023663fb31"). InnerVolumeSpecName "kube-api-access-lpfl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:19:21.133773 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.133745    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c03e1f5a-4afc-4d05-8f22-48023663fb31" (UID: "c03e1f5a-4afc-4d05-8f22-48023663fb31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:19:21.231659 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.231618    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:19:21.231659 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.231652    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c03e1f5a-4afc-4d05-8f22-48023663fb31-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:19:21.231659 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.231663    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpfl7\" (UniqueName: \"kubernetes.io/projected/c03e1f5a-4afc-4d05-8f22-48023663fb31-kube-api-access-lpfl7\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:19:21.231974 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.231674    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c03e1f5a-4afc-4d05-8f22-48023663fb31-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:19:21.404436 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.404337    2581 generic.go:356] "Generic (PLEG): container finished" podID="c03e1f5a-4afc-4d05-8f22-48023663fb31" containerID="8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0" exitCode=0
Sep 18 19:19:21.404436 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.404377    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v" event={"ID":"c03e1f5a-4afc-4d05-8f22-48023663fb31","Type":"ContainerDied","Data":"8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0"}
Sep 18 19:19:21.404436 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.404386    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"
Sep 18 19:19:21.404436 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.404416    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v" event={"ID":"c03e1f5a-4afc-4d05-8f22-48023663fb31","Type":"ContainerDied","Data":"cd7770f67117418e5a44bf4e460e3e21eec38528a004b4d2403829a77fd25386"}
Sep 18 19:19:21.404436 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.404437    2581 scope.go:117] "RemoveContainer" containerID="8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0"
Sep 18 19:19:21.414996 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.414974    2581 scope.go:117] "RemoveContainer" containerID="8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0"
Sep 18 19:19:21.415330 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:19:21.415305    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0\": container with ID starting with 8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0 not found: ID does not exist" containerID="8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0"
Sep 18 19:19:21.415406 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.415344    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0"} err="failed to get container status \"8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0\": rpc error: code = NotFound desc = could not find container \"8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0\": container with ID starting with 8ccfaae62ebee0b4647f93996d59f5eacebc3753feba8240c67215306c9b60e0 not found: ID does not exist"
Sep 18 19:19:21.466312 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.466266    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"]
Sep 18 19:19:21.489191 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:21.489144    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-790775-7cffc648b9-c887v"]
Sep 18 19:19:22.314557 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:22.314517    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c03e1f5a-4afc-4d05-8f22-48023663fb31" path="/var/lib/kubelet/pods/c03e1f5a-4afc-4d05-8f22-48023663fb31/volumes"
Sep 18 19:19:58.465609 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:19:58.465565    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"]
Sep 18 19:20:13.486123 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.486079    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z" podUID="0d0eb56a-f9ff-420e-ae37-7930e1412215" containerName="main" containerID="cri-o://9dfca032ee446fdb4cfbc02c5b665d115d0fd6494d0f1e7ca1885ab2f64a01ad" gracePeriod=45
Sep 18 19:20:13.721142 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.721110    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d0eb56a-f9ff-420e-ae37-7930e1412215" containerID="9dfca032ee446fdb4cfbc02c5b665d115d0fd6494d0f1e7ca1885ab2f64a01ad" exitCode=0
Sep 18 19:20:13.721337 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.721186    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z" event={"ID":"0d0eb56a-f9ff-420e-ae37-7930e1412215","Type":"ContainerDied","Data":"9dfca032ee446fdb4cfbc02c5b665d115d0fd6494d0f1e7ca1885ab2f64a01ad"}
Sep 18 19:20:13.748810 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.748739    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:20:13.859421 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.859369    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qrtq\" (UniqueName: \"kubernetes.io/projected/0d0eb56a-f9ff-420e-ae37-7930e1412215-kube-api-access-4qrtq\") pod \"0d0eb56a-f9ff-420e-ae37-7930e1412215\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") "
Sep 18 19:20:13.859421 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.859429    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-tmp-dir\") pod \"0d0eb56a-f9ff-420e-ae37-7930e1412215\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") "
Sep 18 19:20:13.859662 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.859582    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-model-cache\") pod \"0d0eb56a-f9ff-420e-ae37-7930e1412215\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") "
Sep 18 19:20:13.859662 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.859617    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-home\") pod \"0d0eb56a-f9ff-420e-ae37-7930e1412215\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") "
Sep 18 19:20:13.859662 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.859648    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-dshm\") pod \"0d0eb56a-f9ff-420e-ae37-7930e1412215\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") "
Sep 18 19:20:13.859838 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.859677    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d0eb56a-f9ff-420e-ae37-7930e1412215-tls-certs\") pod \"0d0eb56a-f9ff-420e-ae37-7930e1412215\" (UID: \"0d0eb56a-f9ff-420e-ae37-7930e1412215\") "
Sep 18 19:20:13.859838 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.859768    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0d0eb56a-f9ff-420e-ae37-7930e1412215" (UID: "0d0eb56a-f9ff-420e-ae37-7930e1412215"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:20:13.859954 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.859889    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-home" (OuterVolumeSpecName: "home") pod "0d0eb56a-f9ff-420e-ae37-7930e1412215" (UID: "0d0eb56a-f9ff-420e-ae37-7930e1412215"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:20:13.859954 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.859905    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-model-cache" (OuterVolumeSpecName: "model-cache") pod "0d0eb56a-f9ff-420e-ae37-7930e1412215" (UID: "0d0eb56a-f9ff-420e-ae37-7930e1412215"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:20:13.860143 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.860088    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:13.860143 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.860107    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:13.860143 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.860117    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:13.861983 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.861947    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-dshm" (OuterVolumeSpecName: "dshm") pod "0d0eb56a-f9ff-420e-ae37-7930e1412215" (UID: "0d0eb56a-f9ff-420e-ae37-7930e1412215"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:20:13.862091 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.862000    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d0eb56a-f9ff-420e-ae37-7930e1412215-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0d0eb56a-f9ff-420e-ae37-7930e1412215" (UID: "0d0eb56a-f9ff-420e-ae37-7930e1412215"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:20:13.862091 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.862007    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d0eb56a-f9ff-420e-ae37-7930e1412215-kube-api-access-4qrtq" (OuterVolumeSpecName: "kube-api-access-4qrtq") pod "0d0eb56a-f9ff-420e-ae37-7930e1412215" (UID: "0d0eb56a-f9ff-420e-ae37-7930e1412215"). InnerVolumeSpecName "kube-api-access-4qrtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:20:13.960867 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.960828    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qrtq\" (UniqueName: \"kubernetes.io/projected/0d0eb56a-f9ff-420e-ae37-7930e1412215-kube-api-access-4qrtq\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:13.960867 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.960861    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d0eb56a-f9ff-420e-ae37-7930e1412215-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:13.960867 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:13.960870    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d0eb56a-f9ff-420e-ae37-7930e1412215-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:14.729090 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:14.729055    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"
Sep 18 19:20:14.729534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:14.729089    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z" event={"ID":"0d0eb56a-f9ff-420e-ae37-7930e1412215","Type":"ContainerDied","Data":"c5343a02b7bf781bd30ba5d66b3e0d1159938cbeb74ffc3467984a63b99cae30"}
Sep 18 19:20:14.729534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:14.729139    2581 scope.go:117] "RemoveContainer" containerID="9dfca032ee446fdb4cfbc02c5b665d115d0fd6494d0f1e7ca1885ab2f64a01ad"
Sep 18 19:20:14.752281 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:14.752247    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"]
Sep 18 19:20:14.763502 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:14.763470    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-f8cdfc/ipp-route-f8cdfc-kserve-5d9bdf59f5-llf4z"]
Sep 18 19:20:16.314385 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:16.314288    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d0eb56a-f9ff-420e-ae37-7930e1412215" path="/var/lib/kubelet/pods/0d0eb56a-f9ff-420e-ae37-7930e1412215/volumes"
Sep 18 19:20:45.688000 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:45.687956    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"]
Sep 18 19:20:45.688591 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:45.688182    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy" containerID="cri-o://fccccd04e83c0381a5338df646fabe91217377585e6c94b1dec13f7002a7835a" gracePeriod=30
Sep 18 19:20:48.470148 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:48.470103    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 19:20:48.470628 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:48.470190    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 19:20:50.942721 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:50.942667    2581 generic.go:356] "Generic (PLEG): container finished" podID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerID="fccccd04e83c0381a5338df646fabe91217377585e6c94b1dec13f7002a7835a" exitCode=0
Sep 18 19:20:50.943099 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:50.942744    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" event={"ID":"a74eac0a-7c51-4404-a978-2416ae665fbf","Type":"ContainerDied","Data":"fccccd04e83c0381a5338df646fabe91217377585e6c94b1dec13f7002a7835a"}
Sep 18 19:20:50.943099 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:50.942807    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9" event={"ID":"a74eac0a-7c51-4404-a978-2416ae665fbf","Type":"ContainerDied","Data":"87ffb83e87fcbfc5cbce03352203c50939fc3a321b01db74af28c2f5fd7a994d"}
Sep 18 19:20:50.943099 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:50.942826    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87ffb83e87fcbfc5cbce03352203c50939fc3a321b01db74af28c2f5fd7a994d"
Sep 18 19:20:50.943099 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:50.942842    2581 scope.go:117] "RemoveContainer" containerID="18de322fd7586879e864a3c696daad7fc5bf938bcd11302707f3197731df33da"
Sep 18 19:20:50.952920 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:50.952683    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:20:51.000249 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000205    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-podinfo\") pod \"a74eac0a-7c51-4404-a978-2416ae665fbf\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") "
Sep 18 19:20:51.000451 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000259    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-socket\") pod \"a74eac0a-7c51-4404-a978-2416ae665fbf\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") "
Sep 18 19:20:51.000451 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000283    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-data\") pod \"a74eac0a-7c51-4404-a978-2416ae665fbf\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") "
Sep 18 19:20:51.000451 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000340    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-credential-socket\") pod \"a74eac0a-7c51-4404-a978-2416ae665fbf\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") "
Sep 18 19:20:51.000451 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000373    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-certs\") pod \"a74eac0a-7c51-4404-a978-2416ae665fbf\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") "
Sep 18 19:20:51.000451 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000409    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-envoy\") pod \"a74eac0a-7c51-4404-a978-2416ae665fbf\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") "
Sep 18 19:20:51.000451 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000446    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-token\") pod \"a74eac0a-7c51-4404-a978-2416ae665fbf\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") "
Sep 18 19:20:51.000811 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000476    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c57qp\" (UniqueName: \"kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-kube-api-access-c57qp\") pod \"a74eac0a-7c51-4404-a978-2416ae665fbf\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") "
Sep 18 19:20:51.000811 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000593    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a74eac0a-7c51-4404-a978-2416ae665fbf-istiod-ca-cert\") pod \"a74eac0a-7c51-4404-a978-2416ae665fbf\" (UID: \"a74eac0a-7c51-4404-a978-2416ae665fbf\") "
Sep 18 19:20:51.000811 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000594    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a74eac0a-7c51-4404-a978-2416ae665fbf" (UID: "a74eac0a-7c51-4404-a978-2416ae665fbf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:20:51.000811 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000602    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-data" (OuterVolumeSpecName: "istio-data") pod "a74eac0a-7c51-4404-a978-2416ae665fbf" (UID: "a74eac0a-7c51-4404-a978-2416ae665fbf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:20:51.000811 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000657    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a74eac0a-7c51-4404-a978-2416ae665fbf" (UID: "a74eac0a-7c51-4404-a978-2416ae665fbf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:20:51.001037 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000785    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a74eac0a-7c51-4404-a978-2416ae665fbf" (UID: "a74eac0a-7c51-4404-a978-2416ae665fbf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:20:51.001037 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.000989    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:51.001037 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.001012    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:51.001037 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.001027    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:51.001241 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.001025    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a74eac0a-7c51-4404-a978-2416ae665fbf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a74eac0a-7c51-4404-a978-2416ae665fbf" (UID: "a74eac0a-7c51-4404-a978-2416ae665fbf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:20:51.001241 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.001042    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:51.002892 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.002851    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a74eac0a-7c51-4404-a978-2416ae665fbf" (UID: "a74eac0a-7c51-4404-a978-2416ae665fbf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:20:51.003054 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.003029    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-kube-api-access-c57qp" (OuterVolumeSpecName: "kube-api-access-c57qp") pod "a74eac0a-7c51-4404-a978-2416ae665fbf" (UID: "a74eac0a-7c51-4404-a978-2416ae665fbf"). InnerVolumeSpecName "kube-api-access-c57qp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:20:51.003140 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.003080    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a74eac0a-7c51-4404-a978-2416ae665fbf" (UID: "a74eac0a-7c51-4404-a978-2416ae665fbf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:20:51.003256 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.003236    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-token" (OuterVolumeSpecName: "istio-token") pod "a74eac0a-7c51-4404-a978-2416ae665fbf" (UID: "a74eac0a-7c51-4404-a978-2416ae665fbf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:20:51.101831 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.101787    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:51.101831 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.101820    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:51.101831 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.101830    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c57qp\" (UniqueName: \"kubernetes.io/projected/a74eac0a-7c51-4404-a978-2416ae665fbf-kube-api-access-c57qp\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:51.101831 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.101848    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a74eac0a-7c51-4404-a978-2416ae665fbf-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:51.102125 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.101858    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a74eac0a-7c51-4404-a978-2416ae665fbf-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:51.950078 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:51.950044    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"
Sep 18 19:20:52.026930 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:52.026889    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"]
Sep 18 19:20:52.030184 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:52.030149    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-f8cdfc-openshift-default-8586f775cd-lnkf9"]
Sep 18 19:20:52.314804 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:52.314720    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" path="/var/lib/kubelet/pods/a74eac0a-7c51-4404-a978-2416ae665fbf/volumes"
Sep 18 19:20:58.118341 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.118296    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"]
Sep 18 19:20:58.118883 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.118580    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9" podUID="cb52db18-5ddf-45f6-80e6-f8c7004eca81" containerName="maas-api" containerID="cri-o://f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903" gracePeriod=30
Sep 18 19:20:58.394745 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.394694    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:20:58.476960 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.476925    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptq2q\" (UniqueName: \"kubernetes.io/projected/cb52db18-5ddf-45f6-80e6-f8c7004eca81-kube-api-access-ptq2q\") pod \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") "
Sep 18 19:20:58.477171 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.476981    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb52db18-5ddf-45f6-80e6-f8c7004eca81-openshift-service-ca\") pod \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") "
Sep 18 19:20:58.477171 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.477021    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-metrics-certs\") pod \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") "
Sep 18 19:20:58.477171 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.477062    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-maas-api-tls\") pod \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\" (UID: \"cb52db18-5ddf-45f6-80e6-f8c7004eca81\") "
Sep 18 19:20:58.477469 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.477436    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb52db18-5ddf-45f6-80e6-f8c7004eca81-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cb52db18-5ddf-45f6-80e6-f8c7004eca81" (UID: "cb52db18-5ddf-45f6-80e6-f8c7004eca81"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:20:58.479566 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.479528    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cb52db18-5ddf-45f6-80e6-f8c7004eca81" (UID: "cb52db18-5ddf-45f6-80e6-f8c7004eca81"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:20:58.479566 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.479545    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cb52db18-5ddf-45f6-80e6-f8c7004eca81" (UID: "cb52db18-5ddf-45f6-80e6-f8c7004eca81"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:20:58.479819 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.479588    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb52db18-5ddf-45f6-80e6-f8c7004eca81-kube-api-access-ptq2q" (OuterVolumeSpecName: "kube-api-access-ptq2q") pod "cb52db18-5ddf-45f6-80e6-f8c7004eca81" (UID: "cb52db18-5ddf-45f6-80e6-f8c7004eca81"). InnerVolumeSpecName "kube-api-access-ptq2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:20:58.578510 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.578464    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:58.578510 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.578501    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptq2q\" (UniqueName: \"kubernetes.io/projected/cb52db18-5ddf-45f6-80e6-f8c7004eca81-kube-api-access-ptq2q\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:58.578510 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.578513    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb52db18-5ddf-45f6-80e6-f8c7004eca81-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:58.578510 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.578521    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb52db18-5ddf-45f6-80e6-f8c7004eca81-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:20:58.993080 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.993047    2581 generic.go:356] "Generic (PLEG): container finished" podID="cb52db18-5ddf-45f6-80e6-f8c7004eca81" containerID="f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903" exitCode=0
Sep 18 19:20:58.993314 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.993095    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"
Sep 18 19:20:58.993314 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.993136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9" event={"ID":"cb52db18-5ddf-45f6-80e6-f8c7004eca81","Type":"ContainerDied","Data":"f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903"}
Sep 18 19:20:58.993314 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.993200    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9" event={"ID":"cb52db18-5ddf-45f6-80e6-f8c7004eca81","Type":"ContainerDied","Data":"44635a6d91105392f873fd431b83f94ea0cd177cfdbc94a6c8a7d1b09022b389"}
Sep 18 19:20:58.993314 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:58.993231    2581 scope.go:117] "RemoveContainer" containerID="f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903"
Sep 18 19:20:59.003423 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:59.003400    2581 scope.go:117] "RemoveContainer" containerID="f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903"
Sep 18 19:20:59.003755 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:20:59.003729    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903\": container with ID starting with f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903 not found: ID does not exist" containerID="f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903"
Sep 18 19:20:59.003836 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:59.003766    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903"} err="failed to get container status \"f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903\": rpc error: code = NotFound desc = could not find container \"f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903\": container with ID starting with f1ca16d8808ea9e4313419be7afa4e1677daaa16c2b29bce48582a8f0812b903 not found: ID does not exist"
Sep 18 19:20:59.094953 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:59.094914    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"]
Sep 18 19:20:59.109231 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:20:59.109197    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8f9642-747c949d7c-hcbw9"]
Sep 18 19:21:00.313612 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:00.313575    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb52db18-5ddf-45f6-80e6-f8c7004eca81" path="/var/lib/kubelet/pods/cb52db18-5ddf-45f6-80e6-f8c7004eca81/volumes"
Sep 18 19:21:10.503032 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:10.502990    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"]
Sep 18 19:21:10.503482 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:10.503225    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy" containerID="cri-o://942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f" gracePeriod=30
Sep 18 19:21:12.319155 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:12.319116    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 19:21:12.319547 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:12.319194    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 19:21:15.764183 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.764108    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:21:15.838476 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.838441    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvpmr\" (UniqueName: \"kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-kube-api-access-vvpmr\") pod \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") "
Sep 18 19:21:15.838663 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.838488    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-data\") pod \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") "
Sep 18 19:21:15.838663 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.838543    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-podinfo\") pod \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") "
Sep 18 19:21:15.838663 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.838565    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-envoy\") pod \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") "
Sep 18 19:21:15.838663 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.838611    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-socket\") pod \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") "
Sep 18 19:21:15.838663 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.838654    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-credential-socket\") pod \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") "
Sep 18 19:21:15.838962 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.838691    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-token\") pod \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") "
Sep 18 19:21:15.838962 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.838753    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-certs\") pod \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") "
Sep 18 19:21:15.838962 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.838782    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istiod-ca-cert\") pod \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\" (UID: \"c7e0084c-1782-41b0-8a4b-09337b0b20e8\") "
Sep 18 19:21:15.838962 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.838925    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "c7e0084c-1782-41b0-8a4b-09337b0b20e8" (UID: "c7e0084c-1782-41b0-8a4b-09337b0b20e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:15.839212 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.839139    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:15.839268 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.839207    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c7e0084c-1782-41b0-8a4b-09337b0b20e8" (UID: "c7e0084c-1782-41b0-8a4b-09337b0b20e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:21:15.839395 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.839343    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c7e0084c-1782-41b0-8a4b-09337b0b20e8" (UID: "c7e0084c-1782-41b0-8a4b-09337b0b20e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:15.839395 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.839338    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c7e0084c-1782-41b0-8a4b-09337b0b20e8" (UID: "c7e0084c-1782-41b0-8a4b-09337b0b20e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:15.839581 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.839479    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c7e0084c-1782-41b0-8a4b-09337b0b20e8" (UID: "c7e0084c-1782-41b0-8a4b-09337b0b20e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:15.841300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.841252    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "c7e0084c-1782-41b0-8a4b-09337b0b20e8" (UID: "c7e0084c-1782-41b0-8a4b-09337b0b20e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:15.841459 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.841435    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-kube-api-access-vvpmr" (OuterVolumeSpecName: "kube-api-access-vvpmr") pod "c7e0084c-1782-41b0-8a4b-09337b0b20e8" (UID: "c7e0084c-1782-41b0-8a4b-09337b0b20e8"). InnerVolumeSpecName "kube-api-access-vvpmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:15.841511 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.841432    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c7e0084c-1782-41b0-8a4b-09337b0b20e8" (UID: "c7e0084c-1782-41b0-8a4b-09337b0b20e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:21:15.841511 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.841439    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c7e0084c-1782-41b0-8a4b-09337b0b20e8" (UID: "c7e0084c-1782-41b0-8a4b-09337b0b20e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:15.940528 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.940487    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:15.940528 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.940523    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:15.940528 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.940534    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:15.940829 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.940544    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvpmr\" (UniqueName: \"kubernetes.io/projected/c7e0084c-1782-41b0-8a4b-09337b0b20e8-kube-api-access-vvpmr\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:15.940829 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.940557    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:15.940829 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.940567    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:15.940829 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.940578    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:15.940829 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:15.940588    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e0084c-1782-41b0-8a4b-09337b0b20e8-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:16.103483 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.103383    2581 generic.go:356] "Generic (PLEG): container finished" podID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerID="942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f" exitCode=0
Sep 18 19:21:16.103483 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.103439    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"
Sep 18 19:21:16.103805 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.103472    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" event={"ID":"c7e0084c-1782-41b0-8a4b-09337b0b20e8","Type":"ContainerDied","Data":"942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f"}
Sep 18 19:21:16.103805 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.103542    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn" event={"ID":"c7e0084c-1782-41b0-8a4b-09337b0b20e8","Type":"ContainerDied","Data":"02c8a156310aaf06ff6a64767d5044e867a977655bb56b97495146b97cf70d16"}
Sep 18 19:21:16.103805 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.103572    2581 scope.go:117] "RemoveContainer" containerID="942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f"
Sep 18 19:21:16.115526 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.115439    2581 scope.go:117] "RemoveContainer" containerID="0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a"
Sep 18 19:21:16.125179 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.125156    2581 scope.go:117] "RemoveContainer" containerID="942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f"
Sep 18 19:21:16.125515 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:21:16.125497    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f\": container with ID starting with 942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f not found: ID does not exist" containerID="942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f"
Sep 18 19:21:16.125557 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.125524    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f"} err="failed to get container status \"942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f\": rpc error: code = NotFound desc = could not find container \"942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f\": container with ID starting with 942d1d508720a19e46f6df023adb7e59214f957425f7207c67482463d232f36f not found: ID does not exist"
Sep 18 19:21:16.125557 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.125542    2581 scope.go:117] "RemoveContainer" containerID="0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a"
Sep 18 19:21:16.125830 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:21:16.125812    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a\": container with ID starting with 0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a not found: ID does not exist" containerID="0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a"
Sep 18 19:21:16.125907 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.125834    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a"} err="failed to get container status \"0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a\": rpc error: code = NotFound desc = could not find container \"0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a\": container with ID starting with 0d994bc13269a96285dd9d01a3432e3471f9224c4c0eef0f7eb24985fd8dfe6a not found: ID does not exist"
Sep 18 19:21:16.158657 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.158624    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"]
Sep 18 19:21:16.166509 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.166478    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-8f9642-openshift-default-64bbbb9d86-h8ldn"]
Sep 18 19:21:16.314199 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:16.314155    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" path="/var/lib/kubelet/pods/c7e0084c-1782-41b0-8a4b-09337b0b20e8/volumes"
Sep 18 19:21:21.742239 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:21.742188    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst"]
Sep 18 19:21:21.748952 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:21.748901    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-feb80d-29829315-qpfst"]
Sep 18 19:21:22.313664 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:22.313623    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56d2a12b-9c14-4d59-a5c6-c30b05f83333" path="/var/lib/kubelet/pods/56d2a12b-9c14-4d59-a5c6-c30b05f83333/volumes"
Sep 18 19:21:26.610215 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:26.610179    2581 scope.go:117] "RemoveContainer" containerID="b5068f0e5bd11765e646c191a883a78ac7870f68fa828912fa2bca4eb87ff32a"
Sep 18 19:21:54.269335 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:54.269294    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"]
Sep 18 19:21:54.270000 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:54.269579    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy" containerID="cri-o://d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6" gracePeriod=30
Sep 18 19:21:57.746113 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:57.746071    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84944d87d7-bfdh2"]
Sep 18 19:21:57.746585 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:57.746264    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84944d87d7-bfdh2" podUID="95a94070-857b-4c5d-9d60-7885f761c121" containerName="manager" containerID="cri-o://887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e" gracePeriod=30
Sep 18 19:21:58.001428 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.001349    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 19:21:58.049517 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.049475    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx4xc\" (UniqueName: \"kubernetes.io/projected/95a94070-857b-4c5d-9d60-7885f761c121-kube-api-access-vx4xc\") pod \"95a94070-857b-4c5d-9d60-7885f761c121\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") "
Sep 18 19:21:58.049517 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.049520    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-metrics-certs\") pod \"95a94070-857b-4c5d-9d60-7885f761c121\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") "
Sep 18 19:21:58.049782 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.049614    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-webhook-cert\") pod \"95a94070-857b-4c5d-9d60-7885f761c121\" (UID: \"95a94070-857b-4c5d-9d60-7885f761c121\") "
Sep 18 19:21:58.052032 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.051992    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a94070-857b-4c5d-9d60-7885f761c121-kube-api-access-vx4xc" (OuterVolumeSpecName: "kube-api-access-vx4xc") pod "95a94070-857b-4c5d-9d60-7885f761c121" (UID: "95a94070-857b-4c5d-9d60-7885f761c121"). InnerVolumeSpecName "kube-api-access-vx4xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:58.052203 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.052176    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "95a94070-857b-4c5d-9d60-7885f761c121" (UID: "95a94070-857b-4c5d-9d60-7885f761c121"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:21:58.052436 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.052411    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "95a94070-857b-4c5d-9d60-7885f761c121" (UID: "95a94070-857b-4c5d-9d60-7885f761c121"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:21:58.150640 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.150591    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-webhook-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:58.150640 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.150629    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx4xc\" (UniqueName: \"kubernetes.io/projected/95a94070-857b-4c5d-9d60-7885f761c121-kube-api-access-vx4xc\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:58.150640 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.150638    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95a94070-857b-4c5d-9d60-7885f761c121-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:58.351866 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.351831    2581 generic.go:356] "Generic (PLEG): container finished" podID="95a94070-857b-4c5d-9d60-7885f761c121" containerID="887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e" exitCode=0
Sep 18 19:21:58.352023 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.351879    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84944d87d7-bfdh2"
Sep 18 19:21:58.352023 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.351918    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84944d87d7-bfdh2" event={"ID":"95a94070-857b-4c5d-9d60-7885f761c121","Type":"ContainerDied","Data":"887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e"}
Sep 18 19:21:58.352023 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.351971    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84944d87d7-bfdh2" event={"ID":"95a94070-857b-4c5d-9d60-7885f761c121","Type":"ContainerDied","Data":"de8fe2f5857a805e4664691ebdce5d252fbfe8532857c3a07f45cad1ba6a0cc9"}
Sep 18 19:21:58.352023 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.351995    2581 scope.go:117] "RemoveContainer" containerID="887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e"
Sep 18 19:21:58.361968 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.361945    2581 scope.go:117] "RemoveContainer" containerID="887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e"
Sep 18 19:21:58.362286 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:21:58.362258    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e\": container with ID starting with 887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e not found: ID does not exist" containerID="887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e"
Sep 18 19:21:58.362351 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.362296    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e"} err="failed to get container status \"887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e\": rpc error: code = NotFound desc = could not find container \"887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e\": container with ID starting with 887859fcd5c8cf8d3ed62162242b11c15d30bc7e780c782b24b9ec64d1c4a66e not found: ID does not exist"
Sep 18 19:21:58.382933 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.382888    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84944d87d7-bfdh2"]
Sep 18 19:21:58.396116 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:58.396076    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84944d87d7-bfdh2"]
Sep 18 19:21:59.532195 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.532165    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:21:59.662807 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.662684    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-podinfo\") pod \"afd77893-0ff8-4b2e-95fa-7dca11d72330\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") "
Sep 18 19:21:59.662807 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.662777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-token\") pod \"afd77893-0ff8-4b2e-95fa-7dca11d72330\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") "
Sep 18 19:21:59.662807 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.662797    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-socket\") pod \"afd77893-0ff8-4b2e-95fa-7dca11d72330\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") "
Sep 18 19:21:59.663120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.662847    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-data\") pod \"afd77893-0ff8-4b2e-95fa-7dca11d72330\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") "
Sep 18 19:21:59.663120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.662872    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-envoy\") pod \"afd77893-0ff8-4b2e-95fa-7dca11d72330\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") "
Sep 18 19:21:59.663120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.662894    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnwxj\" (UniqueName: \"kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-kube-api-access-dnwxj\") pod \"afd77893-0ff8-4b2e-95fa-7dca11d72330\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") "
Sep 18 19:21:59.663120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.662945    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-certs\") pod \"afd77893-0ff8-4b2e-95fa-7dca11d72330\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") "
Sep 18 19:21:59.663120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.662987    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-credential-socket\") pod \"afd77893-0ff8-4b2e-95fa-7dca11d72330\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") "
Sep 18 19:21:59.663120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.663015    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afd77893-0ff8-4b2e-95fa-7dca11d72330-istiod-ca-cert\") pod \"afd77893-0ff8-4b2e-95fa-7dca11d72330\" (UID: \"afd77893-0ff8-4b2e-95fa-7dca11d72330\") "
Sep 18 19:21:59.663120 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.663074    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "afd77893-0ff8-4b2e-95fa-7dca11d72330" (UID: "afd77893-0ff8-4b2e-95fa-7dca11d72330"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:59.663604 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.663145    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-data" (OuterVolumeSpecName: "istio-data") pod "afd77893-0ff8-4b2e-95fa-7dca11d72330" (UID: "afd77893-0ff8-4b2e-95fa-7dca11d72330"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:59.663604 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.663319    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-data\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:59.663604 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.663332    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "afd77893-0ff8-4b2e-95fa-7dca11d72330" (UID: "afd77893-0ff8-4b2e-95fa-7dca11d72330"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:59.663604 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.663340    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:59.663604 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.663466    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "afd77893-0ff8-4b2e-95fa-7dca11d72330" (UID: "afd77893-0ff8-4b2e-95fa-7dca11d72330"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:59.663604 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.663635    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afd77893-0ff8-4b2e-95fa-7dca11d72330-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "afd77893-0ff8-4b2e-95fa-7dca11d72330" (UID: "afd77893-0ff8-4b2e-95fa-7dca11d72330"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:21:59.665250 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.665222    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-token" (OuterVolumeSpecName: "istio-token") pod "afd77893-0ff8-4b2e-95fa-7dca11d72330" (UID: "afd77893-0ff8-4b2e-95fa-7dca11d72330"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:59.665399 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.665372    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-kube-api-access-dnwxj" (OuterVolumeSpecName: "kube-api-access-dnwxj") pod "afd77893-0ff8-4b2e-95fa-7dca11d72330" (UID: "afd77893-0ff8-4b2e-95fa-7dca11d72330"). InnerVolumeSpecName "kube-api-access-dnwxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:59.665520 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.665503    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "afd77893-0ff8-4b2e-95fa-7dca11d72330" (UID: "afd77893-0ff8-4b2e-95fa-7dca11d72330"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:21:59.665673 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.665654    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "afd77893-0ff8-4b2e-95fa-7dca11d72330" (UID: "afd77893-0ff8-4b2e-95fa-7dca11d72330"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:59.764599 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.764554    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-workload-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:59.764599 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.764587    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-credential-socket\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:59.764599 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.764598    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afd77893-0ff8-4b2e-95fa-7dca11d72330-istiod-ca-cert\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:59.764599 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.764606    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-podinfo\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:59.764913 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.764614    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-token\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:59.764913 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.764621    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afd77893-0ff8-4b2e-95fa-7dca11d72330-istio-envoy\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:21:59.764913 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:21:59.764628    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnwxj\" (UniqueName: \"kubernetes.io/projected/afd77893-0ff8-4b2e-95fa-7dca11d72330-kube-api-access-dnwxj\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:22:00.315541 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.315492    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95a94070-857b-4c5d-9d60-7885f761c121" path="/var/lib/kubelet/pods/95a94070-857b-4c5d-9d60-7885f761c121/volumes"
Sep 18 19:22:00.369353 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.369306    2581 generic.go:356] "Generic (PLEG): container finished" podID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerID="d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6" exitCode=0
Sep 18 19:22:00.369557 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.369373    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"
Sep 18 19:22:00.369557 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.369390    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" event={"ID":"afd77893-0ff8-4b2e-95fa-7dca11d72330","Type":"ContainerDied","Data":"d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6"}
Sep 18 19:22:00.369557 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.369461    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq" event={"ID":"afd77893-0ff8-4b2e-95fa-7dca11d72330","Type":"ContainerDied","Data":"bf3b108eb90b2be7d55fb80e44935e2ce3ab0ca7a1c450edf6eb1b1e7ae04d47"}
Sep 18 19:22:00.369557 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.369490    2581 scope.go:117] "RemoveContainer" containerID="d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6"
Sep 18 19:22:00.385374 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.385341    2581 scope.go:117] "RemoveContainer" containerID="70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104"
Sep 18 19:22:00.396726 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.396680    2581 scope.go:117] "RemoveContainer" containerID="d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6"
Sep 18 19:22:00.397124 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:22:00.397100    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6\": container with ID starting with d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6 not found: ID does not exist" containerID="d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6"
Sep 18 19:22:00.397230 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.397138    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6"} err="failed to get container status \"d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6\": rpc error: code = NotFound desc = could not find container \"d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6\": container with ID starting with d46fb58b4064ff1bb7161e8b5ed95c18f16df0890e8fdb43db944512439a48d6 not found: ID does not exist"
Sep 18 19:22:00.397230 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.397164    2581 scope.go:117] "RemoveContainer" containerID="70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104"
Sep 18 19:22:00.397569 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:22:00.397540    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104\": container with ID starting with 70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104 not found: ID does not exist" containerID="70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104"
Sep 18 19:22:00.397645 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.397580    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104"} err="failed to get container status \"70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104\": rpc error: code = NotFound desc = could not find container \"70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104\": container with ID starting with 70398cdf1fd0c9e0aff3d9757dac653855c17ab6dc2732ff1b52f1ac366ce104 not found: ID does not exist"
Sep 18 19:22:00.407448 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.407412    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"]
Sep 18 19:22:00.414938 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:00.414902    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-19944d-openshift-default-6bf6bf4dcb-n7nnq"]
Sep 18 19:22:02.313264 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:22:02.313231    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" path="/var/lib/kubelet/pods/afd77893-0ff8-4b2e-95fa-7dca11d72330/volumes"
Sep 18 19:23:24.674489 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674395    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84944d87d7-srfxj"]
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674773    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674786    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674796    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674803    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674812    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d0eb56a-f9ff-420e-ae37-7930e1412215" containerName="main"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674818    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d0eb56a-f9ff-420e-ae37-7930e1412215" containerName="main"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674827    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674832    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674841    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a94070-857b-4c5d-9d60-7885f761c121" containerName="manager"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674846    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a94070-857b-4c5d-9d60-7885f761c121" containerName="manager"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674852    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674856    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674866    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c03e1f5a-4afc-4d05-8f22-48023663fb31" containerName="maas-api"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674871    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c03e1f5a-4afc-4d05-8f22-48023663fb31" containerName="maas-api"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674876    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb52db18-5ddf-45f6-80e6-f8c7004eca81" containerName="maas-api"
Sep 18 19:23:24.674916 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674881    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb52db18-5ddf-45f6-80e6-f8c7004eca81" containerName="maas-api"
Sep 18 19:23:24.675384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674932    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy"
Sep 18 19:23:24.675384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674941    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy"
Sep 18 19:23:24.675384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674946    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy"
Sep 18 19:23:24.675384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674951    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d0eb56a-f9ff-420e-ae37-7930e1412215" containerName="main"
Sep 18 19:23:24.675384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674958    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb52db18-5ddf-45f6-80e6-f8c7004eca81" containerName="maas-api"
Sep 18 19:23:24.675384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674964    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a94070-857b-4c5d-9d60-7885f761c121" containerName="manager"
Sep 18 19:23:24.675384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674970    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy"
Sep 18 19:23:24.675384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674976    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c03e1f5a-4afc-4d05-8f22-48023663fb31" containerName="maas-api"
Sep 18 19:23:24.675384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674981    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7e0084c-1782-41b0-8a4b-09337b0b20e8" containerName="istio-proxy"
Sep 18 19:23:24.675384 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.674986    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy"
Sep 18 19:23:24.677922 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.677905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:24.681915 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.681894    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pcg7z\""
Sep 18 19:23:24.682799 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.682761    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 19:23:24.682923 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.682904    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 19:23:24.682974 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.682909    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 19:23:24.701152 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.701126    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84944d87d7-srfxj"]
Sep 18 19:23:24.715096 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.715062    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1361633-f426-40a3-9135-46bdd4c1d816-metrics-certs\") pod \"maas-controller-84944d87d7-srfxj\" (UID: \"d1361633-f426-40a3-9135-46bdd4c1d816\") " pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:24.715229 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.715112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48nf7\" (UniqueName: \"kubernetes.io/projected/d1361633-f426-40a3-9135-46bdd4c1d816-kube-api-access-48nf7\") pod \"maas-controller-84944d87d7-srfxj\" (UID: \"d1361633-f426-40a3-9135-46bdd4c1d816\") " pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:24.715229 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.715136    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1361633-f426-40a3-9135-46bdd4c1d816-webhook-cert\") pod \"maas-controller-84944d87d7-srfxj\" (UID: \"d1361633-f426-40a3-9135-46bdd4c1d816\") " pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:24.816583 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.816551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48nf7\" (UniqueName: \"kubernetes.io/projected/d1361633-f426-40a3-9135-46bdd4c1d816-kube-api-access-48nf7\") pod \"maas-controller-84944d87d7-srfxj\" (UID: \"d1361633-f426-40a3-9135-46bdd4c1d816\") " pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:24.816814 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.816602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1361633-f426-40a3-9135-46bdd4c1d816-webhook-cert\") pod \"maas-controller-84944d87d7-srfxj\" (UID: \"d1361633-f426-40a3-9135-46bdd4c1d816\") " pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:24.816814 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.816689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1361633-f426-40a3-9135-46bdd4c1d816-metrics-certs\") pod \"maas-controller-84944d87d7-srfxj\" (UID: \"d1361633-f426-40a3-9135-46bdd4c1d816\") " pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:24.822735 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.820025    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1361633-f426-40a3-9135-46bdd4c1d816-webhook-cert\") pod \"maas-controller-84944d87d7-srfxj\" (UID: \"d1361633-f426-40a3-9135-46bdd4c1d816\") " pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:24.822735 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.820078    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1361633-f426-40a3-9135-46bdd4c1d816-metrics-certs\") pod \"maas-controller-84944d87d7-srfxj\" (UID: \"d1361633-f426-40a3-9135-46bdd4c1d816\") " pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:24.829363 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.829319    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48nf7\" (UniqueName: \"kubernetes.io/projected/d1361633-f426-40a3-9135-46bdd4c1d816-kube-api-access-48nf7\") pod \"maas-controller-84944d87d7-srfxj\" (UID: \"d1361633-f426-40a3-9135-46bdd4c1d816\") " pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:24.987520 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:24.987439    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:25.264577 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:25.264388    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84944d87d7-srfxj"]
Sep 18 19:23:25.267011 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:23:25.266977    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1361633_f426_40a3_9135_46bdd4c1d816.slice/crio-0a64c95ffb0b0fa2cbef99fe56f50f2a03a12a8bfe3f0eda9c05360a61bf4157 WatchSource:0}: Error finding container 0a64c95ffb0b0fa2cbef99fe56f50f2a03a12a8bfe3f0eda9c05360a61bf4157: Status 404 returned error can't find the container with id 0a64c95ffb0b0fa2cbef99fe56f50f2a03a12a8bfe3f0eda9c05360a61bf4157
Sep 18 19:23:25.268773 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:25.268752    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:23:25.835145 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:25.835111    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84944d87d7-srfxj" event={"ID":"d1361633-f426-40a3-9135-46bdd4c1d816","Type":"ContainerStarted","Data":"2118178a24b50649e597ddfbfb6efe8b0e91c7b13c3b6a8362382384ca43537a"}
Sep 18 19:23:25.835534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:25.835163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84944d87d7-srfxj" event={"ID":"d1361633-f426-40a3-9135-46bdd4c1d816","Type":"ContainerStarted","Data":"0a64c95ffb0b0fa2cbef99fe56f50f2a03a12a8bfe3f0eda9c05360a61bf4157"}
Sep 18 19:23:25.835534 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:25.835222    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:23:25.862090 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:25.862005    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84944d87d7-srfxj" podStartSLOduration=1.536851448 podStartE2EDuration="1.861991235s" totalImagesPullingTime="325.139787ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:23:25.861259497 +0000 UTC m=+2342.080267074" watchObservedRunningTime="2026-09-18 19:23:25.861991235 +0000 UTC m=+2342.080998811"
Sep 18 19:23:26.660475 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:26.660441    2581 scope.go:117] "RemoveContainer" containerID="fccccd04e83c0381a5338df646fabe91217377585e6c94b1dec13f7002a7835a"
Sep 18 19:23:36.843651 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:23:36.843621    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84944d87d7-srfxj"
Sep 18 19:24:44.618929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:44.618894    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"]
Sep 18 19:24:44.619444 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:44.619128    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x" podUID="7673227a-1079-400b-a86d-83de486404ad" containerName="manager" containerID="cri-o://30742048d4905be0ad2d966600dc6d70eadd2ef7359efa9595b685ecb9ebf602" gracePeriod=10
Sep 18 19:24:45.269224 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:45.269192    2581 generic.go:356] "Generic (PLEG): container finished" podID="7673227a-1079-400b-a86d-83de486404ad" containerID="30742048d4905be0ad2d966600dc6d70eadd2ef7359efa9595b685ecb9ebf602" exitCode=0
Sep 18 19:24:45.269359 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:45.269270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x" event={"ID":"7673227a-1079-400b-a86d-83de486404ad","Type":"ContainerDied","Data":"30742048d4905be0ad2d966600dc6d70eadd2ef7359efa9595b685ecb9ebf602"}
Sep 18 19:24:45.269359 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:45.269349    2581 scope.go:117] "RemoveContainer" containerID="fc2c1f255eae53ca09f821e85fb4cca907152e1ee1abc6f8e16ecc32fcef160b"
Sep 18 19:24:45.867445 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:45.867419    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 19:24:46.013051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.012962    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7b4h\" (UniqueName: \"kubernetes.io/projected/7673227a-1079-400b-a86d-83de486404ad-kube-api-access-v7b4h\") pod \"7673227a-1079-400b-a86d-83de486404ad\" (UID: \"7673227a-1079-400b-a86d-83de486404ad\") "
Sep 18 19:24:46.013051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.013009    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7673227a-1079-400b-a86d-83de486404ad-extensions-socket-volume\") pod \"7673227a-1079-400b-a86d-83de486404ad\" (UID: \"7673227a-1079-400b-a86d-83de486404ad\") "
Sep 18 19:24:46.013445 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.013422    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7673227a-1079-400b-a86d-83de486404ad-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7673227a-1079-400b-a86d-83de486404ad" (UID: "7673227a-1079-400b-a86d-83de486404ad"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:24:46.015166 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.015141    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7673227a-1079-400b-a86d-83de486404ad-kube-api-access-v7b4h" (OuterVolumeSpecName: "kube-api-access-v7b4h") pod "7673227a-1079-400b-a86d-83de486404ad" (UID: "7673227a-1079-400b-a86d-83de486404ad"). InnerVolumeSpecName "kube-api-access-v7b4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:24:46.114424 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.114388    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7b4h\" (UniqueName: \"kubernetes.io/projected/7673227a-1079-400b-a86d-83de486404ad-kube-api-access-v7b4h\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:24:46.114424 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.114419    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7673227a-1079-400b-a86d-83de486404ad-extensions-socket-volume\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:24:46.275623 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.275537    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"
Sep 18 19:24:46.275623 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.275569    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x" event={"ID":"7673227a-1079-400b-a86d-83de486404ad","Type":"ContainerDied","Data":"182cc1344b34d1dfc8b48c3938379371661680ee414aa9f36047c7d302f08d8d"}
Sep 18 19:24:46.275623 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.275619    2581 scope.go:117] "RemoveContainer" containerID="30742048d4905be0ad2d966600dc6d70eadd2ef7359efa9595b685ecb9ebf602"
Sep 18 19:24:46.306016 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.305982    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"]
Sep 18 19:24:46.314897 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:46.314874    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f244x"]
Sep 18 19:24:48.312931 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:24:48.312899    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7673227a-1079-400b-a86d-83de486404ad" path="/var/lib/kubelet/pods/7673227a-1079-400b-a86d-83de486404ad/volumes"
Sep 18 19:25:45.303344 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.303307    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"]
Sep 18 19:25:45.303795 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.303667    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7673227a-1079-400b-a86d-83de486404ad" containerName="manager"
Sep 18 19:25:45.303795 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.303678    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7673227a-1079-400b-a86d-83de486404ad" containerName="manager"
Sep 18 19:25:45.303795 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.303693    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy"
Sep 18 19:25:45.303795 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.303712    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a74eac0a-7c51-4404-a978-2416ae665fbf" containerName="istio-proxy"
Sep 18 19:25:45.303795 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.303724    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy"
Sep 18 19:25:45.303795 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.303733    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="afd77893-0ff8-4b2e-95fa-7dca11d72330" containerName="istio-proxy"
Sep 18 19:25:45.303795 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.303749    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7673227a-1079-400b-a86d-83de486404ad" containerName="manager"
Sep 18 19:25:45.303795 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.303754    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7673227a-1079-400b-a86d-83de486404ad" containerName="manager"
Sep 18 19:25:45.304089 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.303842    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7673227a-1079-400b-a86d-83de486404ad" containerName="manager"
Sep 18 19:25:45.307147 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.307125    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"
Sep 18 19:25:45.315894 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.315871    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-grlk2\""
Sep 18 19:25:45.336425 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.336392    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"]
Sep 18 19:25:45.418053 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.418021    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkrtw\" (UniqueName: \"kubernetes.io/projected/9060c671-849c-4036-b3c4-45a86d78b233-kube-api-access-qkrtw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wdlx6\" (UID: \"9060c671-849c-4036-b3c4-45a86d78b233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"
Sep 18 19:25:45.418241 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.418099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9060c671-849c-4036-b3c4-45a86d78b233-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wdlx6\" (UID: \"9060c671-849c-4036-b3c4-45a86d78b233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"
Sep 18 19:25:45.518675 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.518643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkrtw\" (UniqueName: \"kubernetes.io/projected/9060c671-849c-4036-b3c4-45a86d78b233-kube-api-access-qkrtw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wdlx6\" (UID: \"9060c671-849c-4036-b3c4-45a86d78b233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"
Sep 18 19:25:45.518887 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.518697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9060c671-849c-4036-b3c4-45a86d78b233-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wdlx6\" (UID: \"9060c671-849c-4036-b3c4-45a86d78b233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"
Sep 18 19:25:45.519107 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.519087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9060c671-849c-4036-b3c4-45a86d78b233-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wdlx6\" (UID: \"9060c671-849c-4036-b3c4-45a86d78b233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"
Sep 18 19:25:45.530069 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.530044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkrtw\" (UniqueName: \"kubernetes.io/projected/9060c671-849c-4036-b3c4-45a86d78b233-kube-api-access-qkrtw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wdlx6\" (UID: \"9060c671-849c-4036-b3c4-45a86d78b233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"
Sep 18 19:25:45.617565 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.617461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"
Sep 18 19:25:45.913202 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:45.913175    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"]
Sep 18 19:25:45.915935 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:25:45.915906    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9060c671_849c_4036_b3c4_45a86d78b233.slice/crio-3a22d7b5b01bce5aea45c04f8cc1d70a1db637180d747c21eff87517a4876617 WatchSource:0}: Error finding container 3a22d7b5b01bce5aea45c04f8cc1d70a1db637180d747c21eff87517a4876617: Status 404 returned error can't find the container with id 3a22d7b5b01bce5aea45c04f8cc1d70a1db637180d747c21eff87517a4876617
Sep 18 19:25:46.580046 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:46.580008    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6" event={"ID":"9060c671-849c-4036-b3c4-45a86d78b233","Type":"ContainerStarted","Data":"2f70c22f29f09c76094b19874d45a0f33a0ea556c469f96a1cfc0f35d21f2613"}
Sep 18 19:25:46.580496 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:46.580060    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6" event={"ID":"9060c671-849c-4036-b3c4-45a86d78b233","Type":"ContainerStarted","Data":"3a22d7b5b01bce5aea45c04f8cc1d70a1db637180d747c21eff87517a4876617"}
Sep 18 19:25:46.580496 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:46.580142    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"
Sep 18 19:25:46.631086 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:46.631032    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6" podStartSLOduration=1.631014108 podStartE2EDuration="1.631014108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:25:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:25:46.627395571 +0000 UTC m=+2482.846403149" watchObservedRunningTime="2026-09-18 19:25:46.631014108 +0000 UTC m=+2482.850021685"
Sep 18 19:25:57.587536 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:25:57.587506    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wdlx6"
Sep 18 19:26:29.156591 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.156502    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"]
Sep 18 19:26:29.157171 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.157151    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7673227a-1079-400b-a86d-83de486404ad" containerName="manager"
Sep 18 19:26:29.160816 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.160797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.168188 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.168165    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-73f386a2-kserve-self-signed-certs\""
Sep 18 19:26:29.196682 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.196654    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"]
Sep 18 19:26:29.336624 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.336587    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xdpq\" (UniqueName: \"kubernetes.io/projected/080a8160-5947-4ef1-a033-ae4b29267a65-kube-api-access-4xdpq\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.336846 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.336632    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-home\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.336846 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.336677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-dshm\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.336846 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.336715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-model-cache\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.336846 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.336755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080a8160-5947-4ef1-a033-ae4b29267a65-tls-certs\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.336846 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.336786    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-tmp-dir\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.437974 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.437936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xdpq\" (UniqueName: \"kubernetes.io/projected/080a8160-5947-4ef1-a033-ae4b29267a65-kube-api-access-4xdpq\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.438178 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.437991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-home\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.438178 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.438044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-dshm\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.438178 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.438070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-model-cache\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.438178 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.438118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080a8160-5947-4ef1-a033-ae4b29267a65-tls-certs\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.438178 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.438165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-tmp-dir\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.438463 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.438441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-home\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.438463 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.438458    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-model-cache\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.438609 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.438586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-tmp-dir\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.440571 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.440544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-dshm\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.440905 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.440889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080a8160-5947-4ef1-a033-ae4b29267a65-tls-certs\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.457908 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.457881    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xdpq\" (UniqueName: \"kubernetes.io/projected/080a8160-5947-4ef1-a033-ae4b29267a65-kube-api-access-4xdpq\") pod \"e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") " pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.470603 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.470577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:29.764371 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.764345    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"]
Sep 18 19:26:29.766379 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:26:29.766350    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod080a8160_5947_4ef1_a033_ae4b29267a65.slice/crio-1dc588878886b89ee8a4e7be29f1e1d9c55050f4e37a90e69f773e8f239fbe69 WatchSource:0}: Error finding container 1dc588878886b89ee8a4e7be29f1e1d9c55050f4e37a90e69f773e8f239fbe69: Status 404 returned error can't find the container with id 1dc588878886b89ee8a4e7be29f1e1d9c55050f4e37a90e69f773e8f239fbe69
Sep 18 19:26:29.802173 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:29.802143    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm" event={"ID":"080a8160-5947-4ef1-a033-ae4b29267a65","Type":"ContainerStarted","Data":"1dc588878886b89ee8a4e7be29f1e1d9c55050f4e37a90e69f773e8f239fbe69"}
Sep 18 19:26:30.808863 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:30.808827    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm" event={"ID":"080a8160-5947-4ef1-a033-ae4b29267a65","Type":"ContainerStarted","Data":"f7d5c8dd94282c92530f89d833cd2457d7b6aa95f62c8251d3effe029dd5b359"}
Sep 18 19:26:30.834051 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:30.833993    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm" podStartSLOduration=1.833977055 podStartE2EDuration="1.833977055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:26:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:26:30.833937256 +0000 UTC m=+2527.052944834" watchObservedRunningTime="2026-09-18 19:26:30.833977055 +0000 UTC m=+2527.052984634"
Sep 18 19:26:39.471640 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:39.471598    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:39.472052 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:39.471659    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:39.484462 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:39.484434    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:26:39.870033 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:26:39.869954    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:27:08.954803 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:08.954764    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"]
Sep 18 19:27:23.973799 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:23.973735    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm" podUID="080a8160-5947-4ef1-a033-ae4b29267a65" containerName="main" containerID="cri-o://f7d5c8dd94282c92530f89d833cd2457d7b6aa95f62c8251d3effe029dd5b359" gracePeriod=45
Sep 18 19:27:24.092985 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.092956    2581 generic.go:356] "Generic (PLEG): container finished" podID="080a8160-5947-4ef1-a033-ae4b29267a65" containerID="f7d5c8dd94282c92530f89d833cd2457d7b6aa95f62c8251d3effe029dd5b359" exitCode=0
Sep 18 19:27:24.093117 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.093029    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm" event={"ID":"080a8160-5947-4ef1-a033-ae4b29267a65","Type":"ContainerDied","Data":"f7d5c8dd94282c92530f89d833cd2457d7b6aa95f62c8251d3effe029dd5b359"}
Sep 18 19:27:24.212929 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.212904    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:27:24.276033 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.275961    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-dshm\") pod \"080a8160-5947-4ef1-a033-ae4b29267a65\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") "
Sep 18 19:27:24.276033 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.275997    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-tmp-dir\") pod \"080a8160-5947-4ef1-a033-ae4b29267a65\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") "
Sep 18 19:27:24.276033 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.276026    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080a8160-5947-4ef1-a033-ae4b29267a65-tls-certs\") pod \"080a8160-5947-4ef1-a033-ae4b29267a65\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") "
Sep 18 19:27:24.276292 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.276063    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-model-cache\") pod \"080a8160-5947-4ef1-a033-ae4b29267a65\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") "
Sep 18 19:27:24.276292 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.276090    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-home\") pod \"080a8160-5947-4ef1-a033-ae4b29267a65\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") "
Sep 18 19:27:24.276292 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.276146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xdpq\" (UniqueName: \"kubernetes.io/projected/080a8160-5947-4ef1-a033-ae4b29267a65-kube-api-access-4xdpq\") pod \"080a8160-5947-4ef1-a033-ae4b29267a65\" (UID: \"080a8160-5947-4ef1-a033-ae4b29267a65\") "
Sep 18 19:27:24.276292 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.276266    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "080a8160-5947-4ef1-a033-ae4b29267a65" (UID: "080a8160-5947-4ef1-a033-ae4b29267a65"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:27:24.276497 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.276334    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-model-cache" (OuterVolumeSpecName: "model-cache") pod "080a8160-5947-4ef1-a033-ae4b29267a65" (UID: "080a8160-5947-4ef1-a033-ae4b29267a65"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:27:24.276497 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.276410    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-home" (OuterVolumeSpecName: "home") pod "080a8160-5947-4ef1-a033-ae4b29267a65" (UID: "080a8160-5947-4ef1-a033-ae4b29267a65"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:27:24.276497 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.276465    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-tmp-dir\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:27:24.276497 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.276482    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-model-cache\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:27:24.278365 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.278339    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/080a8160-5947-4ef1-a033-ae4b29267a65-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "080a8160-5947-4ef1-a033-ae4b29267a65" (UID: "080a8160-5947-4ef1-a033-ae4b29267a65"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:27:24.278477 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.278370    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-dshm" (OuterVolumeSpecName: "dshm") pod "080a8160-5947-4ef1-a033-ae4b29267a65" (UID: "080a8160-5947-4ef1-a033-ae4b29267a65"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:27:24.278477 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.278402    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/080a8160-5947-4ef1-a033-ae4b29267a65-kube-api-access-4xdpq" (OuterVolumeSpecName: "kube-api-access-4xdpq") pod "080a8160-5947-4ef1-a033-ae4b29267a65" (UID: "080a8160-5947-4ef1-a033-ae4b29267a65"). InnerVolumeSpecName "kube-api-access-4xdpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:27:24.377234 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.377203    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xdpq\" (UniqueName: \"kubernetes.io/projected/080a8160-5947-4ef1-a033-ae4b29267a65-kube-api-access-4xdpq\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:27:24.377234 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.377228    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-dshm\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:27:24.377234 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.377237    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080a8160-5947-4ef1-a033-ae4b29267a65-tls-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:27:24.377473 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:24.377246    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080a8160-5947-4ef1-a033-ae4b29267a65-home\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:27:25.099207 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:25.099180    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"
Sep 18 19:27:25.099638 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:25.099220    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm" event={"ID":"080a8160-5947-4ef1-a033-ae4b29267a65","Type":"ContainerDied","Data":"1dc588878886b89ee8a4e7be29f1e1d9c55050f4e37a90e69f773e8f239fbe69"}
Sep 18 19:27:25.099638 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:25.099273    2581 scope.go:117] "RemoveContainer" containerID="f7d5c8dd94282c92530f89d833cd2457d7b6aa95f62c8251d3effe029dd5b359"
Sep 18 19:27:26.713899 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:26.713824    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"]
Sep 18 19:27:26.723999 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:26.723969    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-73f386a2-kserve-6cc56d4b55-j9bkm"]
Sep 18 19:27:28.312750 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:28.312716    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="080a8160-5947-4ef1-a033-ae4b29267a65" path="/var/lib/kubelet/pods/080a8160-5947-4ef1-a033-ae4b29267a65/volumes"
Sep 18 19:27:41.523761 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.523729    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"]
Sep 18 19:27:41.524228 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.524102    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="080a8160-5947-4ef1-a033-ae4b29267a65" containerName="main"
Sep 18 19:27:41.524228 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.524114    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="080a8160-5947-4ef1-a033-ae4b29267a65" containerName="main"
Sep 18 19:27:41.524228 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.524168    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="080a8160-5947-4ef1-a033-ae4b29267a65" containerName="main"
Sep 18 19:27:41.526810 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.526793    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.557147 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.557114    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"]
Sep 18 19:27:41.619411 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.619374    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fa50419-99d6-41ab-8493-2333f90d2af0-maas-api-tls\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.619411 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.619416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxfkn\" (UniqueName: \"kubernetes.io/projected/6fa50419-99d6-41ab-8493-2333f90d2af0-kube-api-access-dxfkn\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.619607 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.619463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fa50419-99d6-41ab-8493-2333f90d2af0-openshift-service-ca\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.619607 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.619534    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fa50419-99d6-41ab-8493-2333f90d2af0-metrics-certs\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.720951 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.720915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fa50419-99d6-41ab-8493-2333f90d2af0-maas-api-tls\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.720951 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.720951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxfkn\" (UniqueName: \"kubernetes.io/projected/6fa50419-99d6-41ab-8493-2333f90d2af0-kube-api-access-dxfkn\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.721169 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.720993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fa50419-99d6-41ab-8493-2333f90d2af0-openshift-service-ca\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.721169 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.721027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fa50419-99d6-41ab-8493-2333f90d2af0-metrics-certs\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.721664 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.721641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fa50419-99d6-41ab-8493-2333f90d2af0-openshift-service-ca\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.723607 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.723586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fa50419-99d6-41ab-8493-2333f90d2af0-maas-api-tls\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.723697 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.723615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fa50419-99d6-41ab-8493-2333f90d2af0-metrics-certs\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.734341 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.734321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxfkn\" (UniqueName: \"kubernetes.io/projected/6fa50419-99d6-41ab-8493-2333f90d2af0-kube-api-access-dxfkn\") pod \"maas-api-66f68574b7-mz5zr\" (UID: \"6fa50419-99d6-41ab-8493-2333f90d2af0\") " pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:41.836931 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:41.836835    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:42.118330 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:42.118304    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"]
Sep 18 19:27:42.120455 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:27:42.120428    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fa50419_99d6_41ab_8493_2333f90d2af0.slice/crio-a2f8d8694f7cab9822d6f5e0bd52e7560ddf0da40d5a1c146a646e1ddc231d3f WatchSource:0}: Error finding container a2f8d8694f7cab9822d6f5e0bd52e7560ddf0da40d5a1c146a646e1ddc231d3f: Status 404 returned error can't find the container with id a2f8d8694f7cab9822d6f5e0bd52e7560ddf0da40d5a1c146a646e1ddc231d3f
Sep 18 19:27:42.201610 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:42.201579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr" event={"ID":"6fa50419-99d6-41ab-8493-2333f90d2af0","Type":"ContainerStarted","Data":"a2f8d8694f7cab9822d6f5e0bd52e7560ddf0da40d5a1c146a646e1ddc231d3f"}
Sep 18 19:27:43.208900 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:43.208857    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr" event={"ID":"6fa50419-99d6-41ab-8493-2333f90d2af0","Type":"ContainerStarted","Data":"65ea0c6e0c751933556bb49a667b5e5819ea3aa5cc23eaf30005b2400956da70"}
Sep 18 19:27:43.209335 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:43.208972    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:43.234300 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:43.234248    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr" podStartSLOduration=1.819616084 podStartE2EDuration="2.23423154s" totalImagesPullingTime="414.615456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:27:43.230570962 +0000 UTC m=+2599.449578539" watchObservedRunningTime="2026-09-18 19:27:43.23423154 +0000 UTC m=+2599.453239110"
Sep 18 19:27:49.220040 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:49.220011    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-66f68574b7-mz5zr"
Sep 18 19:27:49.359062 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:49.359023    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-868977f559-ggcg7"]
Sep 18 19:27:49.359250 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:49.359227    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7" podUID="8d7111d8-8281-4c7c-b55e-8310699a7bb0" containerName="maas-api" containerID="cri-o://f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f" gracePeriod=30
Sep 18 19:27:50.609979 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.609953    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 19:27:50.703863 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.703784    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-maas-api-tls\") pod \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") "
Sep 18 19:27:50.703863 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.703861    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-metrics-certs\") pod \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") "
Sep 18 19:27:50.704060 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.703900    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktwpv\" (UniqueName: \"kubernetes.io/projected/8d7111d8-8281-4c7c-b55e-8310699a7bb0-kube-api-access-ktwpv\") pod \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") "
Sep 18 19:27:50.704060 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.703931    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d7111d8-8281-4c7c-b55e-8310699a7bb0-openshift-service-ca\") pod \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\" (UID: \"8d7111d8-8281-4c7c-b55e-8310699a7bb0\") "
Sep 18 19:27:50.704321 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.704281    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d7111d8-8281-4c7c-b55e-8310699a7bb0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8d7111d8-8281-4c7c-b55e-8310699a7bb0" (UID: "8d7111d8-8281-4c7c-b55e-8310699a7bb0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:27:50.706105 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.706078    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8d7111d8-8281-4c7c-b55e-8310699a7bb0" (UID: "8d7111d8-8281-4c7c-b55e-8310699a7bb0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:27:50.706229 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.706203    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d7111d8-8281-4c7c-b55e-8310699a7bb0-kube-api-access-ktwpv" (OuterVolumeSpecName: "kube-api-access-ktwpv") pod "8d7111d8-8281-4c7c-b55e-8310699a7bb0" (UID: "8d7111d8-8281-4c7c-b55e-8310699a7bb0"). InnerVolumeSpecName "kube-api-access-ktwpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:27:50.706273 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.706211    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8d7111d8-8281-4c7c-b55e-8310699a7bb0" (UID: "8d7111d8-8281-4c7c-b55e-8310699a7bb0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:27:50.805150 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.805120    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-maas-api-tls\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:27:50.805150 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.805144    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d7111d8-8281-4c7c-b55e-8310699a7bb0-metrics-certs\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:27:50.805150 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.805153    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktwpv\" (UniqueName: \"kubernetes.io/projected/8d7111d8-8281-4c7c-b55e-8310699a7bb0-kube-api-access-ktwpv\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:27:50.805387 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:50.805163    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d7111d8-8281-4c7c-b55e-8310699a7bb0-openshift-service-ca\") on node \"ip-10-0-136-52.ec2.internal\" DevicePath \"\""
Sep 18 19:27:51.255616 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:51.255585    2581 generic.go:356] "Generic (PLEG): container finished" podID="8d7111d8-8281-4c7c-b55e-8310699a7bb0" containerID="f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f" exitCode=0
Sep 18 19:27:51.255814 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:51.255634    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7"
Sep 18 19:27:51.255814 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:51.255645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7" event={"ID":"8d7111d8-8281-4c7c-b55e-8310699a7bb0","Type":"ContainerDied","Data":"f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f"}
Sep 18 19:27:51.255814 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:51.255688    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-868977f559-ggcg7" event={"ID":"8d7111d8-8281-4c7c-b55e-8310699a7bb0","Type":"ContainerDied","Data":"0b4ee04920aae328bc2d344bc7cdcff66b8ae4d8ca57735f396bd08f25855ce0"}
Sep 18 19:27:51.255814 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:51.255731    2581 scope.go:117] "RemoveContainer" containerID="f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f"
Sep 18 19:27:51.265571 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:51.265549    2581 scope.go:117] "RemoveContainer" containerID="f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f"
Sep 18 19:27:51.265894 ip-10-0-136-52 kubenswrapper[2581]: E0918 19:27:51.265866    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f\": container with ID starting with f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f not found: ID does not exist" containerID="f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f"
Sep 18 19:27:51.265977 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:51.265899    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f"} err="failed to get container status \"f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f\": rpc error: code = NotFound desc = could not find container \"f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f\": container with ID starting with f721e7360cf7eb6b77cc2232d217c746e899f30b43227a5df6803dce2fb1522f not found: ID does not exist"
Sep 18 19:27:51.287380 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:51.287356    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-868977f559-ggcg7"]
Sep 18 19:27:51.310119 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:51.310097    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-868977f559-ggcg7"]
Sep 18 19:27:52.312636 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:27:52.312608    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d7111d8-8281-4c7c-b55e-8310699a7bb0" path="/var/lib/kubelet/pods/8d7111d8-8281-4c7c-b55e-8310699a7bb0/volumes"
Sep 18 19:28:19.096410 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:19.096378    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h59gr_6b51048b-0988-46e6-aec0-08e4061ad991/manager/0.log"
Sep 18 19:28:19.224283 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:19.224253    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bb74d_680220d0-27f4-4b0c-9693-c685c08a1b67/manager/0.log"
Sep 18 19:28:19.400740 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:19.400645    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84944d87d7-srfxj_d1361633-f426-40a3-9135-46bdd4c1d816/manager/0.log"
Sep 18 19:28:20.204891 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:20.204868    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699fb7c78d-mcfsk_4a923edf-fadc-4e15-95d7-e09df8794c9e/copy-manifests/0.log"
Sep 18 19:28:20.229950 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:20.229925    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699fb7c78d-mcfsk_4a923edf-fadc-4e15-95d7-e09df8794c9e/manager/0.log"
Sep 18 19:28:20.604370 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:20.604300    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-66f68574b7-mz5zr_6fa50419-99d6-41ab-8493-2333f90d2af0/maas-api/0.log"
Sep 18 19:28:21.094305 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:21.094275    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-7wlw4_e8c94a57-1680-4607-a4e5-5f7efc70e391/postgres/0.log"
Sep 18 19:28:22.070484 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:22.070447    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-xq5pf_b978db62-0178-43d7-8ec2-a001bcd2ce96/manager/0.log"
Sep 18 19:28:22.188148 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:22.188122    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-gdwhd_80562742-a164-47ef-87d7-60f2b67ef4db/manager/0.log"
Sep 18 19:28:22.436326 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:22.436298    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-wdlx6_9060c671-849c-4036-b3c4-45a86d78b233/manager/0.log"
Sep 18 19:28:22.774395 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:22.774310    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-ddjfl_081fc10e-a177-4bbc-977e-41978480fb31/manager/0.log"
Sep 18 19:28:23.528769 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:23.528740    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cd84dd4ff-z94r4_fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6/kube-auth-proxy/0.log"
Sep 18 19:28:23.652978 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:23.652954    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jlxh4_a2982db7-81a2-4b40-88b5-2abdac9e0152/istio-proxy/1.log"
Sep 18 19:28:23.925596 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:23.925572    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwd2p_c9c0d59a-eab3-4afb-ab9a-8ee23b012266/payload-processing/0.log"
Sep 18 19:28:24.440925 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:24.440891    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-tmqnp_e810a820-b28d-48db-ab41-d8578f22fb47/main/0.log"
Sep 18 19:28:24.787081 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:24.787008    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8m5fh_57505a2c-7af7-4616-9047-ae9ee25f6c2d/main/0.log"
Sep 18 19:28:31.427284 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.427245    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qnj9f/must-gather-gc96l"]
Sep 18 19:28:31.427821 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.427802    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d7111d8-8281-4c7c-b55e-8310699a7bb0" containerName="maas-api"
Sep 18 19:28:31.427910 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.427824    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d7111d8-8281-4c7c-b55e-8310699a7bb0" containerName="maas-api"
Sep 18 19:28:31.427967 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.427938    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d7111d8-8281-4c7c-b55e-8310699a7bb0" containerName="maas-api"
Sep 18 19:28:31.431074 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.431054    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qnj9f/must-gather-gc96l"
Sep 18 19:28:31.435010 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.434986    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qnj9f\"/\"default-dockercfg-9tcdj\""
Sep 18 19:28:31.435333 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.435317    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qnj9f\"/\"openshift-service-ca.crt\""
Sep 18 19:28:31.436079 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.436067    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qnj9f\"/\"kube-root-ca.crt\""
Sep 18 19:28:31.467270 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.467244    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qnj9f/must-gather-gc96l"]
Sep 18 19:28:31.548364 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.548325    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dls72\" (UniqueName: \"kubernetes.io/projected/5d9dc704-0d7e-47b1-be05-1348afbe785b-kube-api-access-dls72\") pod \"must-gather-gc96l\" (UID: \"5d9dc704-0d7e-47b1-be05-1348afbe785b\") " pod="openshift-must-gather-qnj9f/must-gather-gc96l"
Sep 18 19:28:31.548533 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.548375    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5d9dc704-0d7e-47b1-be05-1348afbe785b-must-gather-output\") pod \"must-gather-gc96l\" (UID: \"5d9dc704-0d7e-47b1-be05-1348afbe785b\") " pod="openshift-must-gather-qnj9f/must-gather-gc96l"
Sep 18 19:28:31.649243 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.649208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dls72\" (UniqueName: \"kubernetes.io/projected/5d9dc704-0d7e-47b1-be05-1348afbe785b-kube-api-access-dls72\") pod \"must-gather-gc96l\" (UID: \"5d9dc704-0d7e-47b1-be05-1348afbe785b\") " pod="openshift-must-gather-qnj9f/must-gather-gc96l"
Sep 18 19:28:31.649455 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.649264    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5d9dc704-0d7e-47b1-be05-1348afbe785b-must-gather-output\") pod \"must-gather-gc96l\" (UID: \"5d9dc704-0d7e-47b1-be05-1348afbe785b\") " pod="openshift-must-gather-qnj9f/must-gather-gc96l"
Sep 18 19:28:31.649592 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.649576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5d9dc704-0d7e-47b1-be05-1348afbe785b-must-gather-output\") pod \"must-gather-gc96l\" (UID: \"5d9dc704-0d7e-47b1-be05-1348afbe785b\") " pod="openshift-must-gather-qnj9f/must-gather-gc96l"
Sep 18 19:28:31.660235 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.660216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dls72\" (UniqueName: \"kubernetes.io/projected/5d9dc704-0d7e-47b1-be05-1348afbe785b-kube-api-access-dls72\") pod \"must-gather-gc96l\" (UID: \"5d9dc704-0d7e-47b1-be05-1348afbe785b\") " pod="openshift-must-gather-qnj9f/must-gather-gc96l"
Sep 18 19:28:31.740596 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:31.740530    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qnj9f/must-gather-gc96l"
Sep 18 19:28:32.020422 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:32.020398    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qnj9f/must-gather-gc96l"]
Sep 18 19:28:32.022122 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:28:32.022095    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d9dc704_0d7e_47b1_be05_1348afbe785b.slice/crio-8d2b3c73eba35c00d9515258537e0fcefdf83348b409bd305dae02aac3825df8 WatchSource:0}: Error finding container 8d2b3c73eba35c00d9515258537e0fcefdf83348b409bd305dae02aac3825df8: Status 404 returned error can't find the container with id 8d2b3c73eba35c00d9515258537e0fcefdf83348b409bd305dae02aac3825df8
Sep 18 19:28:32.024010 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:32.023995    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:28:32.461814 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:32.461774    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qnj9f/must-gather-gc96l" event={"ID":"5d9dc704-0d7e-47b1-be05-1348afbe785b","Type":"ContainerStarted","Data":"8d2b3c73eba35c00d9515258537e0fcefdf83348b409bd305dae02aac3825df8"}
Sep 18 19:28:33.473100 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:33.473061    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qnj9f/must-gather-gc96l" event={"ID":"5d9dc704-0d7e-47b1-be05-1348afbe785b","Type":"ContainerStarted","Data":"7aeb9b85128639cd53d3f37ec5ff1e0bb6ed768d4f87b884a9f1bddd0118e090"}
Sep 18 19:28:33.473573 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:33.473113    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qnj9f/must-gather-gc96l" event={"ID":"5d9dc704-0d7e-47b1-be05-1348afbe785b","Type":"ContainerStarted","Data":"5b27ff816636a2aac8a9760c3c05e1c0426331d8a4f4d05396530d24dfec94eb"}
Sep 18 19:28:33.509510 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:33.509448    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qnj9f/must-gather-gc96l" podStartSLOduration=1.6744735039999998 podStartE2EDuration="2.509427561s" totalImagesPullingTime="834.954057ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:28:33.508382867 +0000 UTC m=+2649.727390446" watchObservedRunningTime="2026-09-18 19:28:33.509427561 +0000 UTC m=+2649.728435138"
Sep 18 19:28:35.215633 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:35.215606    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zsrbq_1dcfcf8b-4bd5-4919-9315-26fe6852ba7d/global-pull-secret-syncer/0.log"
Sep 18 19:28:35.311560 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:35.311532    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bxzsl_cab91e29-3ede-43e0-9209-0945d73869a7/konnectivity-agent/0.log"
Sep 18 19:28:35.428238 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:35.428203    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-52.ec2.internal_c29e46335904b8ccc50d8f973acddf9a/haproxy/0.log"
Sep 18 19:28:39.771471 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:39.771435    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-xq5pf_b978db62-0178-43d7-8ec2-a001bcd2ce96/manager/0.log"
Sep 18 19:28:39.821576 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:39.821539    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-gdwhd_80562742-a164-47ef-87d7-60f2b67ef4db/manager/0.log"
Sep 18 19:28:39.947178 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:39.947146    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-wdlx6_9060c671-849c-4036-b3c4-45a86d78b233/manager/0.log"
Sep 18 19:28:40.085050 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:40.084970    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-ddjfl_081fc10e-a177-4bbc-977e-41978480fb31/manager/0.log"
Sep 18 19:28:42.369435 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:42.369390    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-q8mj9_a7fa0c11-eafd-4961-a6ab-2d03d0e571db/cluster-monitoring-operator/0.log"
Sep 18 19:28:42.698480 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:42.698442    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rwrc6_761dd490-1d88-43f1-b0be-6aa71c53415b/node-exporter/0.log"
Sep 18 19:28:42.727354 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:42.727304    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rwrc6_761dd490-1d88-43f1-b0be-6aa71c53415b/kube-rbac-proxy/0.log"
Sep 18 19:28:42.763213 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:42.763178    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rwrc6_761dd490-1d88-43f1-b0be-6aa71c53415b/init-textfile/0.log"
Sep 18 19:28:44.436367 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.436307    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"]
Sep 18 19:28:44.443176 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.443149    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.469850 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.469810    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"]
Sep 18 19:28:44.595054 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.595018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-sys\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.595054 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.595054    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-lib-modules\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.595308 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.595084    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-podres\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.595308 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.595128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-proc\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.595308 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.595166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v667l\" (UniqueName: \"kubernetes.io/projected/19621a83-24e9-4b0e-a509-da6a63ef6e6b-kube-api-access-v667l\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.696614 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.696538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-sys\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.696614 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.696574    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-lib-modules\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.696614 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.696595    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-podres\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.696869 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.696620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-proc\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.696869 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.696655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v667l\" (UniqueName: \"kubernetes.io/projected/19621a83-24e9-4b0e-a509-da6a63ef6e6b-kube-api-access-v667l\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.696869 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.696659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-sys\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.696869 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.696689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-lib-modules\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.696869 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.696762    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-podres\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.696869 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.696777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/19621a83-24e9-4b0e-a509-da6a63ef6e6b-proc\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.721419 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.721381    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v667l\" (UniqueName: \"kubernetes.io/projected/19621a83-24e9-4b0e-a509-da6a63ef6e6b-kube-api-access-v667l\") pod \"perf-node-gather-daemonset-z8826\" (UID: \"19621a83-24e9-4b0e-a509-da6a63ef6e6b\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:44.758195 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:44.758157    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:45.064084 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:45.064034    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"]
Sep 18 19:28:45.066566 ip-10-0-136-52 kubenswrapper[2581]: W0918 19:28:45.066509    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod19621a83_24e9_4b0e_a509_da6a63ef6e6b.slice/crio-d26f899f16dd5bc73e596bd1db1ca84428474d8a735a17ceed7fd7c8888f4479 WatchSource:0}: Error finding container d26f899f16dd5bc73e596bd1db1ca84428474d8a735a17ceed7fd7c8888f4479: Status 404 returned error can't find the container with id d26f899f16dd5bc73e596bd1db1ca84428474d8a735a17ceed7fd7c8888f4479
Sep 18 19:28:45.573865 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:45.573816    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826" event={"ID":"19621a83-24e9-4b0e-a509-da6a63ef6e6b","Type":"ContainerStarted","Data":"46c1124e7aab95c0059979b14d04df41a2c980e4d6bc6e507693a6f9001463ea"}
Sep 18 19:28:45.574372 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:45.573890    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826" event={"ID":"19621a83-24e9-4b0e-a509-da6a63ef6e6b","Type":"ContainerStarted","Data":"d26f899f16dd5bc73e596bd1db1ca84428474d8a735a17ceed7fd7c8888f4479"}
Sep 18 19:28:45.574372 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:45.573995    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:45.615338 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:45.615311    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-7r7xw_c9085e34-f699-4aec-9aa7-15b7a35766d7/networking-console-plugin/0.log"
Sep 18 19:28:45.633249 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:45.633187    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826" podStartSLOduration=1.6331639610000002 podStartE2EDuration="1.633163961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:28:45.631302929 +0000 UTC m=+2661.850310517" watchObservedRunningTime="2026-09-18 19:28:45.633163961 +0000 UTC m=+2661.852171540"
Sep 18 19:28:47.740764 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:47.740733    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-k84bn_b14702cc-c97a-45ba-8fac-466e1e76d9fa/volume-data-source-validator/0.log"
Sep 18 19:28:49.201608 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:49.201577    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-smkgk_946a2586-1164-4bdf-9348-22b7089112e0/dns/0.log"
Sep 18 19:28:49.229504 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:49.229478    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-smkgk_946a2586-1164-4bdf-9348-22b7089112e0/kube-rbac-proxy/0.log"
Sep 18 19:28:49.264291 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:49.264262    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bspq5_d2f53f4b-0b86-4f9a-a5f6-0c428539290f/dns-node-resolver/0.log"
Sep 18 19:28:50.027452 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:50.027428    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rqb8l_aa308550-2945-40b0-ba91-87a9c8541846/node-ca/0.log"
Sep 18 19:28:51.562826 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:51.562798    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cd84dd4ff-z94r4_fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6/kube-auth-proxy/0.log"
Sep 18 19:28:51.591270 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:51.591239    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-z8826"
Sep 18 19:28:51.616839 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:51.616805    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jlxh4_a2982db7-81a2-4b40-88b5-2abdac9e0152/istio-proxy/0.log"
Sep 18 19:28:51.627547 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:51.627520    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jlxh4_a2982db7-81a2-4b40-88b5-2abdac9e0152/istio-proxy/1.log"
Sep 18 19:28:51.712121 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:51.712090    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwd2p_c9c0d59a-eab3-4afb-ab9a-8ee23b012266/payload-processing/0.log"
Sep 18 19:28:52.855876 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:52.855756    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z4rg2_b51682e9-4379-4076-a853-ec0306df5d58/serve-healthcheck-canary/0.log"
Sep 18 19:28:53.975436 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:53.975408    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vpnsw_7509e8dd-346c-461c-b2cd-caea0b5f793d/kube-rbac-proxy/0.log"
Sep 18 19:28:54.039168 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:54.039140    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vpnsw_7509e8dd-346c-461c-b2cd-caea0b5f793d/exporter/0.log"
Sep 18 19:28:54.075882 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:54.075765    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vpnsw_7509e8dd-346c-461c-b2cd-caea0b5f793d/extractor/0.log"
Sep 18 19:28:55.427553 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:55.427528    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cd84dd4ff-z94r4_fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6/kube-auth-proxy/0.log"
Sep 18 19:28:55.450855 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:55.450797    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jlxh4_a2982db7-81a2-4b40-88b5-2abdac9e0152/istio-proxy/0.log"
Sep 18 19:28:55.454549 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:55.454529    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jlxh4_a2982db7-81a2-4b40-88b5-2abdac9e0152/istio-proxy/1.log"
Sep 18 19:28:55.512144 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:55.512103    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwd2p_c9c0d59a-eab3-4afb-ab9a-8ee23b012266/payload-processing/0.log"
Sep 18 19:28:56.726136 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:56.726106    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h59gr_6b51048b-0988-46e6-aec0-08e4061ad991/manager/0.log"
Sep 18 19:28:57.052657 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:57.052616    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bb74d_680220d0-27f4-4b0c-9693-c685c08a1b67/manager/0.log"
Sep 18 19:28:57.116275 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:57.116230    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84944d87d7-srfxj_d1361633-f426-40a3-9135-46bdd4c1d816/manager/0.log"
Sep 18 19:28:57.501157 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:57.501121    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699fb7c78d-mcfsk_4a923edf-fadc-4e15-95d7-e09df8794c9e/manager/0.log"
Sep 18 19:28:57.523630 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:57.523601    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699fb7c78d-mcfsk_4a923edf-fadc-4e15-95d7-e09df8794c9e/copy-manifests/0.log"
Sep 18 19:28:57.560262 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:57.560228    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cd84dd4ff-z94r4_fb5e7fe7-96fa-44fb-bb5d-9bc4a449b3f6/kube-auth-proxy/0.log"
Sep 18 19:28:57.590545 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:57.590514    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jlxh4_a2982db7-81a2-4b40-88b5-2abdac9e0152/istio-proxy/0.log"
Sep 18 19:28:57.593692 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:57.593666    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jlxh4_a2982db7-81a2-4b40-88b5-2abdac9e0152/istio-proxy/1.log"
Sep 18 19:28:57.661589 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:57.661557    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwd2p_c9c0d59a-eab3-4afb-ab9a-8ee23b012266/payload-processing/0.log"
Sep 18 19:28:58.800836 ip-10-0-136-52 kubenswrapper[2581]: I0918 19:28:58.800798    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2dlbs_cc6ac864-bc6c-4ef2-b908-b58fc191a053/openshift-lws-operator/0.log"