Sep 15 12:25:32.077101 ip-10-0-141-20 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 12:25:32.451169 ip-10-0-141-20 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 12:25:32.451169 ip-10-0-141-20 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 12:25:32.451169 ip-10-0-141-20 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 12:25:32.451169 ip-10-0-141-20 kubenswrapper[2586]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 12:25:32.451169 ip-10-0-141-20 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 12:25:32.451918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.451820    2586 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 12:25:32.458531 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458506    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:25:32.458531 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458527    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:25:32.458531 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458532    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:25:32.458531 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458536    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:25:32.458531 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458539    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458542    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458546    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458549    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458552    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458555    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458558    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458561    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458563    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458566    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458569    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458572    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458574    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458577    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458587    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458590    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458592    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458595    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458598    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:25:32.458746 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458600    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458603    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458605    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458608    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458610    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458613    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458616    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458618    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458621    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458624    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458627    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458629    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458632    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458636    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458638    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458641    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458644    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458649    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458654    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:25:32.459319 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458658    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458661    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458664    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458667    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458670    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458672    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458675    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458678    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458682    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458685    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458688    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458691    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458693    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458696    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458699    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458702    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458704    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458707    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458709    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458712    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:25:32.459826 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458714    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458717    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458719    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458722    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458725    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458727    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458730    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458733    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458736    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458739    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458741    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458744    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458747    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458750    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458753    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458756    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458758    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458761    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458766    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:25:32.460322 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458768    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458771    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458787    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458790    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458792    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458795    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458800    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458805    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458811    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458817    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458821    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458826    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458829    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458833    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458836    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458839    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458841    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.458844    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460252    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:25:32.460808 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460260    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460263    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460267    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460270    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460273    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460276    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460280    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460283    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460285    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460289    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460291    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460294    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460297    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460299    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460302    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460305    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460308    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460310    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460313    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460316    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460319    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:25:32.461279 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460322    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460325    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460329    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460332    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460335    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460338    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460341    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460343    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460346    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460349    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460351    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460354    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460357    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460360    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460362    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460365    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460368    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460371    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460373    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:25:32.461831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460376    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460378    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460381    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460383    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460386    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460388    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460390    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460393    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460397    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460401    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460404    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460407    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460413    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460416    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460419    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460422    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460425    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460428    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460430    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:25:32.462326 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460433    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460436    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460438    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460441    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460444    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460446    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460449    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460452    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460457    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460461    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460464    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460466    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460469    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460472    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460474    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460477    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460479    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460482    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460485    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:25:32.462804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460488    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460490    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460493    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460495    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460499    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460501    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460504    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460506    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460509    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460512    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460514    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460516    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460519    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460521    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460524    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460527    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460529    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460532    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460534    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460536    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:25:32.463302 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460965    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460972    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460975    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460978    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460982    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460985    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460988    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460991    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460993    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460996    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.460999    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461002    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461005    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461008    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461010    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461013    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461016    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461019    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461021    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461024    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:25:32.463830 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461027    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461029    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461034    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461038    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461041    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461044    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461047    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461050    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461052    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461055    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461057    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461060    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461063    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461066    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461068    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461070    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461073    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461075    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461078    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:25:32.464360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461080    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461083    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461085    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461088    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461091    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461093    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461096    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461098    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461101    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461103    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461106    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461108    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461111    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461114    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461116    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461118    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461121    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461123    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461126    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461128    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:25:32.464843 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461131    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461133    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461136    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461139    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461142    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461144    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461147    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461152    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461154    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461157    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461161    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461164    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461167    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461170    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461173    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461176    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461180    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461183    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461186    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461189    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:25:32.465329 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461191    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461193    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461196    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461198    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461201    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461204    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461206    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461210    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461213    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461216    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461219    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461221    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461224    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461226    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461228    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461231    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461234    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461237    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461239    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.461243    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:25:32.465883 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461322    2586 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461369    2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461378    2586 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461382    2586 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461386    2586 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461390    2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461395    2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461399    2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461403    2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461406    2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461409    2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461413    2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461416    2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461419    2586 flags.go:64] FLAG: --cgroup-root=""
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461422    2586 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461425    2586 flags.go:64] FLAG: --client-ca-file=""
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461429    2586 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461432    2586 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461436    2586 flags.go:64] FLAG: --cluster-domain=""
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461439    2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461442    2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461445    2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461449    2586 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461452    2586 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 12:25:32.466379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461455    2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461458    2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461462    2586 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461465    2586 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461468    2586 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461471    2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461474    2586 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461478    2586 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461482    2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461485    2586 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461488    2586 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461491    2586 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461494    2586 flags.go:64] FLAG: --enable-server="true"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461497    2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461503    2586 flags.go:64] FLAG: --event-burst="100"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461506    2586 flags.go:64] FLAG: --event-qps="50"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461509    2586 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461512    2586 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461515    2586 flags.go:64] FLAG: --eviction-hard=""
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461519    2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461522    2586 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461525    2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461528    2586 flags.go:64] FLAG: --eviction-soft=""
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461531    2586 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461534    2586 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 12:25:32.467002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461537    2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461540    2586 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461543    2586 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461546    2586 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461549    2586 flags.go:64] FLAG: --feature-gates=""
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461552    2586 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461556    2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461559    2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461562    2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461564    2586 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461567    2586 flags.go:64] FLAG: --help="false"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461570    2586 flags.go:64] FLAG: --hostname-override="ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461575    2586 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461578    2586 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461581    2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461585    2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461589    2586 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461591    2586 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461594    2586 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461597    2586 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461600    2586 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461603    2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461607    2586 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461610    2586 flags.go:64] FLAG: --kube-reserved=""
Sep 15 12:25:32.467644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461613    2586 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461615    2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461619    2586 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461621    2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461625    2586 flags.go:64] FLAG: --lock-file=""
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461627    2586 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461630    2586 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461633    2586 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461639    2586 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461642    2586 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461645    2586 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461648    2586 flags.go:64] FLAG: --logging-format="text"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461651    2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461654    2586 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461657    2586 flags.go:64] FLAG: --manifest-url=""
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461659    2586 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461664    2586 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461667    2586 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461671    2586 flags.go:64] FLAG: --max-pods="110"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461673    2586 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461679    2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461682    2586 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461685    2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461688    2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461691    2586 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 12:25:32.468291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461694    2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461701    2586 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461705    2586 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461708    2586 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461711    2586 flags.go:64] FLAG: --pod-cidr=""
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461714    2586 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461719    2586 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461722    2586 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461725    2586 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461728    2586 flags.go:64] FLAG: --port="10250"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461731    2586 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461734    2586 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-006de4276046e854d"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461737    2586 flags.go:64] FLAG: --qos-reserved=""
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461740    2586 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461743    2586 flags.go:64] FLAG: --register-node="true"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461746    2586 flags.go:64] FLAG: --register-with-taints=""
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461750    2586 flags.go:64] FLAG: --registry-burst="10"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461753    2586 flags.go:64] FLAG: --registry-qps="5"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461756    2586 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461759    2586 flags.go:64] FLAG: --reserved-memory=""
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461763    2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461766    2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461769    2586 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461772    2586 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461788    2586 flags.go:64] FLAG: --runonce="false"
Sep 15 12:25:32.468944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461791    2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461794    2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461798    2586 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461802    2586 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461805    2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461809    2586 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461812    2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461815    2586 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461818    2586 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461821    2586 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461824    2586 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461827    2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461831    2586 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461834    2586 flags.go:64] FLAG: --system-cgroups=""
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461837    2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461843    2586 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461846    2586 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461849    2586 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461854    2586 flags.go:64] FLAG: --tls-min-version=""
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461856    2586 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461859    2586 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461862    2586 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461866    2586 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461869    2586 flags.go:64] FLAG: --v="2"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461873    2586 flags.go:64] FLAG: --version="false"
Sep 15 12:25:32.469555 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461881    2586 flags.go:64] FLAG: --vmodule=""
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461885    2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.461888    2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462007    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462012    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462015    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462018    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462021    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462024    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462026    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462031    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462034    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462037    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462039    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462042    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462045    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462047    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462050    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462053    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462056    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:25:32.470193 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462058    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462061    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462064    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462067    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462069    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462072    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462074    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462077    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462080    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462084    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462087    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462090    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462092    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462096    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462098    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462101    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462104    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462106    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462109    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462112    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:25:32.470720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462115    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462117    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462121    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462124    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462127    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462131    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462134    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462137    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462140    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462143    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462147    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462149    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462152    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462155    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462158    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462161    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462164    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462166    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462170    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:25:32.471245 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462173    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462176    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462181    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462183    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462186    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462188    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462191    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462194    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462197    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462199    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462202    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462205    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462207    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462210    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462212    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462216    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462219    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462222    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462224    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462227    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:25:32.471720 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462229    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462232    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462235    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462237    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462240    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462243    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462246    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462248    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462251    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462254    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462257    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462259    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462262    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462264    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462268    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462270    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462274    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462276    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462279    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:25:32.472239 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462281    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462286    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462289    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.462292    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.462912    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.469891    2586 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.469906    2586 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.470610    2586 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.470617    2586 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470675    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470681    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470684    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470688    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470691    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:25:32.472716 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470694    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470697    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470700    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470704    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470706    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470709    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470712    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470715    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470718    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470721    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470724    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470727    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470730    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470733    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470736    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470738    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470741    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470744    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470747    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470749    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:25:32.473079 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470752    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470754    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470757    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470760    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470765    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470767    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470770    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470772    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470790    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470794    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470799    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470802    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470804    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470807    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470810    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470813    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470815    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470818    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470820    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470823    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:25:32.473578 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470826    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470829    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470832    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470835    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470837    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470840    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470843    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470845    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470848    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470850    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470853    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470856    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470858    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470861    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470864    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470867    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470871    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470874    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470877    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470880    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:25:32.474151 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470882    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470885    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470887    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470890    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470893    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470895    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470897    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470900    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470903    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470905    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470908    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470910    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470913    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470915    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470919    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470923    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470926    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470929    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470932    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:25:32.474647 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470935    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470937    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470940    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470943    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470945    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470948    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470951    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470954    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470957    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470961    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470965    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470969    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470972    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470974    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:25:32.475200 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.470977    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.470982    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471113    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471119    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471121    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471125    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471128    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471130    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471133    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471136    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471138    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471141    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471143    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471146    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471149    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:25:32.475545 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471151    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471154    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471156    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471159    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471161    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471164    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471166    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471169    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471172    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471176    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471180    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471183    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471187    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471190    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471192    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471196    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471198    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471201    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471204    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471206    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:25:32.475973 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471209    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471211    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471214    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471216    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471219    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471221    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471224    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471226    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471229    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471232    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471234    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471237    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471239    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471241    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471244    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471247    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471250    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471253    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471255    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471258    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:25:32.476452 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471261    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471264    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471266    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471269    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471272    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471275    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471277    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471280    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471283    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471286    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471289    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471292    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471295    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471299    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471302    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471306    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471308    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471311    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:25:32.476976 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471314    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471317    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471319    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471322    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471324    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471327    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471330    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471332    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471335    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471338    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471340    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471343    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471345    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471348    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471351    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471353    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471356    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471359    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471361    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471364    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:25:32.477410 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471367    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471369    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471372    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471375    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471378    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471381    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471383    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:32.471385    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.471390    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.471506    2586 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.473454    2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.474439    2586 server.go:1019] "Starting client certificate rotation"
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.474541    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.474580    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 12:25:32.477918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.476664    2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 12:25:32.479291 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.479275    2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 12:25:32.491516 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.491494    2586 log.go:25] "Validated CRI v1 runtime API"
Sep 15 12:25:32.494623 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.494596    2586 log.go:25] "Validated CRI v1 image API"
Sep 15 12:25:32.495592 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.495574    2586 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 12:25:32.499736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.499716    2586 fs.go:135] Filesystem UUIDs: map[62273e25-90d9-4222-bfa1-f3795d8f81b5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bb7365e9-8db6-4eb7-b479-eddf931c8f01:/dev/nvme0n1p3]
Sep 15 12:25:32.499804 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.499736    2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 12:25:32.504530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.504509    2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 12:25:32.504738 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.504626    2586 manager.go:216] Machine: {Timestamp:2026-09-15 12:25:32.503409841 +0000 UTC m=+0.315321457 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3126933 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20d6cd5c02be818ba7c0fbfea8fa02 SystemUUID:ec20d6cd-5c02-be81-8ba7-c0fbfea8fa02 BootID:810eb744-37dd-4f0b-836b-d36ab76a5ef3 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:df:ff:60:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:df:ff:60:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:0a:95:87:51:01 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 12:25:32.504738 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.504736    2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 12:25:32.504867 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.504839    2586 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 12:25:32.507708 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.507678    2586 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 12:25:32.507939 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.507712    2586 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-20.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 12:25:32.509040 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.509026    2586 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 12:25:32.509098 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.509047    2586 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 12:25:32.509098 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.509059    2586 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 12:25:32.509731 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.509719    2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 12:25:32.510580 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.510569    2586 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 12:25:32.514328 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.514314    2586 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 12:25:32.514468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.514457    2586 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 12:25:32.514524 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.514497    2586 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 12:25:32.514524 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.514512    2586 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 12:25:32.514614 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.514528    2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 12:25:32.514614 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.514543    2586 kubelet.go:398] "Adding apiserver pod source"
Sep 15 12:25:32.514614 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.514557    2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 12:25:32.514975 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.514961    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 12:25:32.515040 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.514985    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 12:25:32.517853 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.517838    2586 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 12:25:32.519524 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.519510    2586 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 12:25:32.519576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.519540    2586 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 12:25:32.521213 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521201    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 12:25:32.521252 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521223    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 12:25:32.521252 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521232    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 12:25:32.521252 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521238    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 12:25:32.521252 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521246    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 12:25:32.521252 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521252    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 12:25:32.521387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521265    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 12:25:32.521387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521272    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 12:25:32.521387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521278    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 12:25:32.521387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521284    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 12:25:32.521387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521300    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 12:25:32.521387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.521306    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 12:25:32.522264 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.522252    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 12:25:32.522306 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.522265    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 12:25:32.524067 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.524046    2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 12:25:32.524114 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.524049    2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 12:25:32.524114 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.524088    2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-20.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 15 12:25:32.526142 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.526130    2586 server.go:1268] "Started kubelet"
Sep 15 12:25:32.526235 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.526206    2586 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 12:25:32.526286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.526237    2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 12:25:32.526286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.526282    2586 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 12:25:32.527147 ip-10-0-141-20 systemd[1]: Started Kubernetes Kubelet.
Sep 15 12:25:32.530406 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.530382    2586 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 12:25:32.532494 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.532432    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hczbz"
Sep 15 12:25:32.532767 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.532703    2586 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 12:25:32.533483 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.533465    2586 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 12:25:32.537756 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.537735    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 12:25:32.538286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.538263    2586 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 12:25:32.538538 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.538516    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:32.539003 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.538986    2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 12:25:32.539102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.538985    2586 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 12:25:32.539102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539029    2586 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 12:25:32.539194 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539107    2586 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 12:25:32.539194 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539117    2586 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 12:25:32.539257 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539200    2586 factory.go:55] Registering systemd factory
Sep 15 12:25:32.539257 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539218    2586 factory.go:223] Registration of the systemd container factory successfully
Sep 15 12:25:32.539471 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539446    2586 factory.go:153] Registering CRI-O factory
Sep 15 12:25:32.539471 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539463    2586 factory.go:223] Registration of the crio container factory successfully
Sep 15 12:25:32.539608 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539481    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hczbz"
Sep 15 12:25:32.539608 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539510    2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 12:25:32.539608 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539529    2586 factory.go:103] Registering Raw factory
Sep 15 12:25:32.539608 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539545    2586 manager.go:1196] Started watching for new ooms in manager
Sep 15 12:25:32.539903 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.539888    2586 manager.go:349] Starting recovery of all containers
Sep 15 12:25:32.546036 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.546003    2586 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 15 12:25:32.548058 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.548027    2586 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 15 12:25:32.548993 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.546123    2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-20.ec2.internal.18d57e26e89cfd2e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-20.ec2.internal,UID:ip-10-0-141-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-20.ec2.internal,},FirstTimestamp:2026-09-15 12:25:32.526099758 +0000 UTC m=+0.338011190,LastTimestamp:2026-09-15 12:25:32.526099758 +0000 UTC m=+0.338011190,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-20.ec2.internal,}"
Sep 15 12:25:32.552460 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.552320    2586 manager.go:354] Recovery completed
Sep 15 12:25:32.553711 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.553692    2586 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 15 12:25:32.557107 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.557092    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:25:32.561251 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.561231    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:25:32.561347 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.561278    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:25:32.561347 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.561292    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:25:32.561969 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.561955    2586 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 12:25:32.561969 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.561966    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:25:32.562083 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.561967    2586 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 12:25:32.562083 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.562011    2586 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 12:25:32.563195 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.563179    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:25:32.563279 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.563211    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:25:32.563279 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.563226    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:25:32.564127 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.564112    2586 policy_none.go:49] "None policy: Start"
Sep 15 12:25:32.564171 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.564132    2586 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 12:25:32.564171 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.564145    2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 12:25:32.566142 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.566130    2586 policy_none.go:47] "Start"
Sep 15 12:25:32.614527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.611428    2586 manager.go:337] "Starting Device Plugin manager"
Sep 15 12:25:32.614527 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.611464    2586 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 12:25:32.614527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.611475    2586 server.go:87] "Starting device plugin registration server"
Sep 15 12:25:32.614527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.611898    2586 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 12:25:32.614527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.611913    2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 12:25:32.614527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.612021    2586 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 12:25:32.614527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.612110    2586 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 12:25:32.614527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.612120    2586 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 12:25:32.614527 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.612842    2586 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 12:25:32.614527 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.612902    2586 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:32.644393 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.644364    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 12:25:32.645744 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.645727    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 12:25:32.645867 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.645788    2586 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 12:25:32.645867 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.645814    2586 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 12:25:32.645974 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.645902    2586 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 12:25:32.648357 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.648337    2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:25:32.712686 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.712586    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:25:32.713722 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.713706    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:25:32.713807 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.713739    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:25:32.713807 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.713751    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:25:32.713807 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.713790    2586 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.723849 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.723832    2586 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.723932 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.723858    2586 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-20.ec2.internal\": node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:32.743847 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.743822    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:32.746863 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.746847    2586 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal"]
Sep 15 12:25:32.746918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.746909    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:25:32.748031 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.748015    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:25:32.748108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.748044    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:25:32.748108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.748054    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:25:32.750317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.750303    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:25:32.750442 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.750426    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.750479 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.750464    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:25:32.751076 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.751054    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:25:32.751174 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.751083    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:25:32.751174 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.751104    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:25:32.751174 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.751110    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:25:32.751174 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.751127    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:25:32.751174 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.751141    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:25:32.753460 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.753444    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.753535 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.753475    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:25:32.754223 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.754207    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:25:32.754294 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.754233    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:25:32.754294 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.754242    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:25:32.768393 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.768373    2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-20.ec2.internal\" not found" node="ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.772556 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.772541    2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-20.ec2.internal\" not found" node="ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.840838 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.840814    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d46ff4d7787a6b16b6c6c5774307130-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal\" (UID: \"1d46ff4d7787a6b16b6c6c5774307130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.840933 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.840844    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d46ff4d7787a6b16b6c6c5774307130-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal\" (UID: \"1d46ff4d7787a6b16b6c6c5774307130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.840933 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.840866    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4baa257649766985809baa75fc95af2e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-20.ec2.internal\" (UID: \"4baa257649766985809baa75fc95af2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.844911 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.844894    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:32.941744 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.941712    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d46ff4d7787a6b16b6c6c5774307130-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal\" (UID: \"1d46ff4d7787a6b16b6c6c5774307130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.941744 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.941744    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d46ff4d7787a6b16b6c6c5774307130-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal\" (UID: \"1d46ff4d7787a6b16b6c6c5774307130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.941953 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.941761    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4baa257649766985809baa75fc95af2e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-20.ec2.internal\" (UID: \"4baa257649766985809baa75fc95af2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.941953 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.941874    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d46ff4d7787a6b16b6c6c5774307130-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal\" (UID: \"1d46ff4d7787a6b16b6c6c5774307130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.941953 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.941893    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d46ff4d7787a6b16b6c6c5774307130-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal\" (UID: \"1d46ff4d7787a6b16b6c6c5774307130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.941953 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:32.941939    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4baa257649766985809baa75fc95af2e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-20.ec2.internal\" (UID: \"4baa257649766985809baa75fc95af2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:32.944983 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:32.944968    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:33.045761 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.045733    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:33.070957 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.070928    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:33.075928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.075745    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:33.146032 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.145983    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:33.247044 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.246994    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:33.347443 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.347344    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:33.373534 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.373505    2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:25:33.404027 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.403992    2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:25:33.447846 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.447810    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-20.ec2.internal\" not found"
Sep 15 12:25:33.453583 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.453556    2586 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:25:33.474449 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.474415    2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 12:25:33.474624 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.474595    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 12:25:33.474688 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.474635    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 12:25:33.474688 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.474650    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 12:25:33.474688 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.474635    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 12:25:33.515368 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.515336    2586 apiserver.go:52] "Watching apiserver"
Sep 15 12:25:33.531658 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.531626    2586 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 12:25:33.532039 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.532011    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-p5wb9","openshift-network-diagnostics/network-check-target-z4d6q","openshift-ovn-kubernetes/ovnkube-node-nfllx","kube-system/global-pull-secret-syncer-2n7pd","openshift-dns/node-resolver-mqn5j","openshift-image-registry/node-ca-k28bb","openshift-multus/multus-additional-cni-plugins-xl5dx","openshift-multus/network-metrics-daemon-rqsvh","openshift-network-operator/iptables-alerter-2qd9t","kube-system/konnectivity-agent-hcnk6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8","openshift-cluster-node-tuning-operator/tuned-zf9c6"]
Sep 15 12:25:33.535323 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.535298    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.537777 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.537628    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 12:25:33.537977 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.537879    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 12:25:33.537977 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.537896    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:33.537977 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.537951    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 12:25:33.538129 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.537991    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:33.538129 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.538105    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pwcpq\""
Sep 15 12:25:33.538129 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.538125    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 12:25:33.539885 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.538689    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 12:25:33.539885 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.538902    2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:33.541562 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.541540    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.544134 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.544112    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 12:25:33.544287 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.544225    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:33.544377 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.544288    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:33.544377 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.544361    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 12:25:33.544377 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.544370    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 12:25:33.544536 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.544346    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 12:20:33 +0000 UTC" deadline="2028-07-01 05:39:19.421994964 +0000 UTC"
Sep 15 12:25:33.544536 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.544404    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15713h13m45.877597075s"
Sep 15 12:25:33.544536 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.544410    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jx5lj\""
Sep 15 12:25:33.544695 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.544576    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 12:25:33.544956 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.544938    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 12:25:33.545090 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.545074    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 12:25:33.546886 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.546848    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-kubelet\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547024 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.546914    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-slash\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547024 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.546923    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.547024 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.546942    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-var-lib-kubelet\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547024 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.546967    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-etc-kubernetes\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547024 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547008    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dhrv\" (UniqueName: \"kubernetes.io/projected/86349040-e321-4bfa-b5fa-009b7c2a8cf5-kube-api-access-9dhrv\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547035    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21b87a7f-98d6-4b99-928c-284f93a63928-ovn-node-metrics-cert\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547061    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvxqv\" (UniqueName: \"kubernetes.io/projected/21b87a7f-98d6-4b99-928c-284f93a63928-kube-api-access-fvxqv\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547089    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-run-k8s-cni-cncf-io\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547126    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-run-netns\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547181    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-run-multus-certs\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547225    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-var-lib-openvswitch\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547255    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-log-socket\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547287    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-cni-dir\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547312    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-cnibin\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547335    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-run-systemd\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547384    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-var-lib-cni-bin\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547442    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-daemon-config\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547465    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-run-ovn-kubernetes\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547487    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86349040-e321-4bfa-b5fa-009b7c2a8cf5-cni-binary-copy\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547503    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-systemd-units\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547518    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-run-ovn\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547533    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-node-log\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547548    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-cni-bin\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547616    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21b87a7f-98d6-4b99-928c-284f93a63928-ovnkube-config\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547635    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21b87a7f-98d6-4b99-928c-284f93a63928-env-overrides\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.547681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547660    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-run-openvswitch\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547676    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-cni-netd\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547734    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547771    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-system-cni-dir\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547803    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-hostroot\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547827    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-conf-dir\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547851    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-etc-openvswitch\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547890    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21b87a7f-98d6-4b99-928c-284f93a63928-ovnkube-script-lib\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547913    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-run-netns\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547932    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-socket-dir-parent\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.547955    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-var-lib-cni-multus\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.548131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.548004    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7sq9\" (UniqueName: \"kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9\") pod \"network-check-target-z4d6q\" (UID: \"dbd143a3-56aa-4bef-b0a9-792ba54e447e\") " pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:33.549516 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.549495    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.552624 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.552603    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.553253 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.553231    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 12:25:33.553361 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.553269    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 12:25:33.553361 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.553340    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 12:25:33.553361 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.553353    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tzzm6\""
Sep 15 12:25:33.553523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.553393    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 12:25:33.553523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.553404    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 12:25:33.553932 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.553916    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fwljz\""
Sep 15 12:25:33.555740 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.555718    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:33.555865 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.555811    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:33.556729 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.556705    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 12:25:33.556840 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.556755    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 12:25:33.556840 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.556814    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2bb7x\""
Sep 15 12:25:33.558202 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.558186    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.560705 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.560685    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 12:25:33.560797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.560782    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v52jg\""
Sep 15 12:25:33.560910 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.560893    2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 12:25:33.561038 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.561023    2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal"
Sep 15 12:25:33.561093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.561080    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:33.561134 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.561097    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 12:25:33.561404 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.561391    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 12:25:33.563006 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.562984    2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 12:25:33.563612 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.563471    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.564027    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hkhnq\""
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.564091    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.564258    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.565772    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal"]
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.565892    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.572434    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.572446    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cqcf2\""
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.572459    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.572473    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.572583    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 12:25:33.574102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.572934    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 12:25:33.574761 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.574629    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal"]
Sep 15 12:25:33.574989 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.574973    2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 12:25:33.581967 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.581948    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x9pmc\""
Sep 15 12:25:33.632334 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.632300    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sqlxf"
Sep 15 12:25:33.639755 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.639732    2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 12:25:33.639986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.639967    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sqlxf"
Sep 15 12:25:33.649191 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649168    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d549c5b-da45-458e-8451-e0627a418526-cni-binary-copy\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.649306 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649203    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-var-lib-cni-bin\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.649306 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649223    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-daemon-config\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.649306 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649240    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.649306 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649294    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-var-lib-cni-bin\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.649443 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649351    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-run\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.649443 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649381    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-lib-modules\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.649443 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649411    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvbh5\" (UniqueName: \"kubernetes.io/projected/dc48a45a-1993-4cba-a9dc-636f75057239-kube-api-access-cvbh5\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:33.649548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649455    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86349040-e321-4bfa-b5fa-009b7c2a8cf5-cni-binary-copy\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.649548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649485    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-systemd-units\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649510    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-run-ovn\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649536    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-node-log\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649561    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-cni-bin\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649585    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-run-ovn\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649589    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21b87a7f-98d6-4b99-928c-284f93a63928-ovnkube-config\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649633    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-socket-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649635    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-node-log\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649659    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-kubernetes\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649638    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-cni-bin\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649688    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-run-openvswitch\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649588    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-systemd-units\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649713    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-cni-netd\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.649736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649730    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-run-openvswitch\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649747    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-cni-netd\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649757    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649787    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-sysctl-d\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649832    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f5cb386-9dde-4e3c-9122-eb1c6042e0e4-host\") pod \"node-ca-k28bb\" (UID: \"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4\") " pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649853    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649857    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p666k\" (UniqueName: \"kubernetes.io/projected/2f5cb386-9dde-4e3c-9122-eb1c6042e0e4-kube-api-access-p666k\") pod \"node-ca-k28bb\" (UID: \"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4\") " pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649895    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d549c5b-da45-458e-8451-e0627a418526-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649909    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-daemon-config\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649921    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-conf-dir\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649938    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-etc-openvswitch\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649975    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-etc-openvswitch\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649976    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21b87a7f-98d6-4b99-928c-284f93a63928-ovnkube-script-lib\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.649987    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86349040-e321-4bfa-b5fa-009b7c2a8cf5-cni-binary-copy\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650006    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-conf-dir\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650011    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b7ec0a1-a68d-4699-bb5d-c3152791a0ef-tmp-dir\") pod \"node-resolver-mqn5j\" (UID: \"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef\") " pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650062    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/802d16e2-7d79-4df2-bf7b-304ab45e62ce-host-slash\") pod \"iptables-alerter-2qd9t\" (UID: \"802d16e2-7d79-4df2-bf7b-304ab45e62ce\") " pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.650260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650092    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-sys-fs\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650118    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-sysctl-conf\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650143    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-systemd\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650179    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-socket-dir-parent\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650188    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21b87a7f-98d6-4b99-928c-284f93a63928-ovnkube-config\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650208    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7sq9\" (UniqueName: \"kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9\") pod \"network-check-target-z4d6q\" (UID: \"dbd143a3-56aa-4bef-b0a9-792ba54e447e\") " pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650221    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-socket-dir-parent\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650236    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b63ae85-7c59-4a60-9f0e-862df0a82739-tmp\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650263    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650303    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bst84\" (UniqueName: \"kubernetes.io/projected/802d16e2-7d79-4df2-bf7b-304ab45e62ce-kube-api-access-bst84\") pod \"iptables-alerter-2qd9t\" (UID: \"802d16e2-7d79-4df2-bf7b-304ab45e62ce\") " pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650335    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-var-lib-kubelet\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650382    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dhrv\" (UniqueName: \"kubernetes.io/projected/86349040-e321-4bfa-b5fa-009b7c2a8cf5-kube-api-access-9dhrv\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650401    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-var-lib-kubelet\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650410    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvxqv\" (UniqueName: \"kubernetes.io/projected/21b87a7f-98d6-4b99-928c-284f93a63928-kube-api-access-fvxqv\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650441    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pts4t\" (UniqueName: \"kubernetes.io/projected/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-kube-api-access-pts4t\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650469    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/94cf40a2-35af-402c-b86d-f023b2ba9c85-agent-certs\") pod \"konnectivity-agent-hcnk6\" (UID: \"94cf40a2-35af-402c-b86d-f023b2ba9c85\") " pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650469    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21b87a7f-98d6-4b99-928c-284f93a63928-ovnkube-script-lib\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.651052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650498    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d549c5b-da45-458e-8451-e0627a418526-system-cni-dir\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650527    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2d549c5b-da45-458e-8451-e0627a418526-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650556    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-run-netns\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650583    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-run-multus-certs\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650607    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-sys\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650621    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-run-netns\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650647    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-run-multus-certs\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650650    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4v7x\" (UniqueName: \"kubernetes.io/projected/1b63ae85-7c59-4a60-9f0e-862df0a82739-kube-api-access-w4v7x\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650711    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-cni-dir\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650730    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-run-systemd\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650770    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-multus-cni-dir\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650779    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-host\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650852    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-run-systemd\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650849    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/94cf40a2-35af-402c-b86d-f023b2ba9c85-konnectivity-ca\") pod \"konnectivity-agent-hcnk6\" (UID: \"94cf40a2-35af-402c-b86d-f023b2ba9c85\") " pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650908    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-run-ovn-kubernetes\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650927    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ca49e661-dd33-4c7c-9751-408033360944-kubelet-config\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650951    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6b7ec0a1-a68d-4699-bb5d-c3152791a0ef-hosts-file\") pod \"node-resolver-mqn5j\" (UID: \"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef\") " pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.651759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650977    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzcxs\" (UniqueName: \"kubernetes.io/projected/6b7ec0a1-a68d-4699-bb5d-c3152791a0ef-kube-api-access-rzcxs\") pod \"node-resolver-mqn5j\" (UID: \"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef\") " pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.650979    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-run-ovn-kubernetes\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651004    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21b87a7f-98d6-4b99-928c-284f93a63928-env-overrides\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651028    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f5cb386-9dde-4e3c-9122-eb1c6042e0e4-serviceca\") pod \"node-ca-k28bb\" (UID: \"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4\") " pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651046    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2d549c5b-da45-458e-8451-e0627a418526-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651062    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ca49e661-dd33-4c7c-9751-408033360944-dbus\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651082    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8drxh\" (UniqueName: \"kubernetes.io/projected/2d549c5b-da45-458e-8451-e0627a418526-kube-api-access-8drxh\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651110    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-system-cni-dir\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651128    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-hostroot\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651143    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-sysconfig\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651166    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-var-lib-kubelet\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651190    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-etc-selinux\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651207    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-run-netns\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651256    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-var-lib-cni-multus\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651266    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-hostroot\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651281    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-run-netns\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651308    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-system-cni-dir\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.652530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651310    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-kubelet\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651353    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-var-lib-cni-multus\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651365    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-kubelet\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651366    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-slash\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651401    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-host-slash\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651403    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-registration-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651437    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-tuned\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651440    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21b87a7f-98d6-4b99-928c-284f93a63928-env-overrides\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651464    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-etc-kubernetes\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651489    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21b87a7f-98d6-4b99-928c-284f93a63928-ovn-node-metrics-cert\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651532    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-etc-kubernetes\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651558    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-device-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651639    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651663    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d549c5b-da45-458e-8451-e0627a418526-cnibin\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651687    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/802d16e2-7d79-4df2-bf7b-304ab45e62ce-iptables-alerter-script\") pod \"iptables-alerter-2qd9t\" (UID: \"802d16e2-7d79-4df2-bf7b-304ab45e62ce\") " pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651715    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-run-k8s-cni-cncf-io\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651755    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-host-run-k8s-cni-cncf-io\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.653093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651752    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-var-lib-openvswitch\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.653604 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651828    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-var-lib-openvswitch\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.653604 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651843    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-log-socket\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.653604 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651864    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-cnibin\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.653604 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651875    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21b87a7f-98d6-4b99-928c-284f93a63928-log-socket\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.653604 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651883    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-modprobe-d\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.653604 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651938    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86349040-e321-4bfa-b5fa-009b7c2a8cf5-cnibin\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.653604 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.651938    2586 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 12:25:33.655796 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.655767    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21b87a7f-98d6-4b99-928c-284f93a63928-ovn-node-metrics-cert\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.662373 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.662204    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:25:33.662373 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.662228    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:25:33.662373 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.662243    2586 projected.go:196] Error preparing data for projected volume kube-api-access-r7sq9 for pod openshift-network-diagnostics/network-check-target-z4d6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:33.662373 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.662348    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9 podName:dbd143a3-56aa-4bef-b0a9-792ba54e447e nodeName:}" failed. No retries permitted until 2026-09-15 12:25:34.162307924 +0000 UTC m=+1.974219367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r7sq9" (UniqueName: "kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9") pod "network-check-target-z4d6q" (UID: "dbd143a3-56aa-4bef-b0a9-792ba54e447e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:33.664729 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.664691    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dhrv\" (UniqueName: \"kubernetes.io/projected/86349040-e321-4bfa-b5fa-009b7c2a8cf5-kube-api-access-9dhrv\") pod \"multus-p5wb9\" (UID: \"86349040-e321-4bfa-b5fa-009b7c2a8cf5\") " pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.664833 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.664771    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvxqv\" (UniqueName: \"kubernetes.io/projected/21b87a7f-98d6-4b99-928c-284f93a63928-kube-api-access-fvxqv\") pod \"ovnkube-node-nfllx\" (UID: \"21b87a7f-98d6-4b99-928c-284f93a63928\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.698315 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.698279    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d46ff4d7787a6b16b6c6c5774307130.slice/crio-4e8680af98d969b400a284a76907308c3a49a3bf150afb97b3d06bbd84878d26 WatchSource:0}: Error finding container 4e8680af98d969b400a284a76907308c3a49a3bf150afb97b3d06bbd84878d26: Status 404 returned error can't find the container with id 4e8680af98d969b400a284a76907308c3a49a3bf150afb97b3d06bbd84878d26
Sep 15 12:25:33.698956 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.698934    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4baa257649766985809baa75fc95af2e.slice/crio-7241473a18726bdd07b161b0eb6797d0e30c19867ce112e45bfbab49d5f916ea WatchSource:0}: Error finding container 7241473a18726bdd07b161b0eb6797d0e30c19867ce112e45bfbab49d5f916ea: Status 404 returned error can't find the container with id 7241473a18726bdd07b161b0eb6797d0e30c19867ce112e45bfbab49d5f916ea
Sep 15 12:25:33.705051 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.705035    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 12:25:33.752259 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752214    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:33.752470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752266    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d549c5b-da45-458e-8451-e0627a418526-cnibin\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.752470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752295    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/802d16e2-7d79-4df2-bf7b-304ab45e62ce-iptables-alerter-script\") pod \"iptables-alerter-2qd9t\" (UID: \"802d16e2-7d79-4df2-bf7b-304ab45e62ce\") " pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.752470 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.752326    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:33.752470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752337    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d549c5b-da45-458e-8451-e0627a418526-cnibin\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.752470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752326    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-modprobe-d\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.752470 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.752404    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret podName:ca49e661-dd33-4c7c-9751-408033360944 nodeName:}" failed. No retries permitted until 2026-09-15 12:25:34.252384349 +0000 UTC m=+2.064295790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret") pod "global-pull-secret-syncer-2n7pd" (UID: "ca49e661-dd33-4c7c-9751-408033360944") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:33.752470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752425    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d549c5b-da45-458e-8451-e0627a418526-cni-binary-copy\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.752470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752456    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752482    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-run\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752504    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-lib-modules\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752528    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvbh5\" (UniqueName: \"kubernetes.io/projected/dc48a45a-1993-4cba-a9dc-636f75057239-kube-api-access-cvbh5\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752545    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-modprobe-d\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752560    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-socket-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752585    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-kubernetes\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752602    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-run\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752613    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-sysctl-d\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752654    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f5cb386-9dde-4e3c-9122-eb1c6042e0e4-host\") pod \"node-ca-k28bb\" (UID: \"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4\") " pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752607    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752683    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p666k\" (UniqueName: \"kubernetes.io/projected/2f5cb386-9dde-4e3c-9122-eb1c6042e0e4-kube-api-access-p666k\") pod \"node-ca-k28bb\" (UID: \"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4\") " pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752711    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-sysctl-d\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752767    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f5cb386-9dde-4e3c-9122-eb1c6042e0e4-host\") pod \"node-ca-k28bb\" (UID: \"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4\") " pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752877    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-socket-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752713    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d549c5b-da45-458e-8451-e0627a418526-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.752913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752916    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-kubernetes\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752922    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b7ec0a1-a68d-4699-bb5d-c3152791a0ef-tmp-dir\") pod \"node-resolver-mqn5j\" (UID: \"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef\") " pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752960    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-lib-modules\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.752990    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/802d16e2-7d79-4df2-bf7b-304ab45e62ce-host-slash\") pod \"iptables-alerter-2qd9t\" (UID: \"802d16e2-7d79-4df2-bf7b-304ab45e62ce\") " pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753022    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-sys-fs\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753065    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-sysctl-conf\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753092    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/802d16e2-7d79-4df2-bf7b-304ab45e62ce-host-slash\") pod \"iptables-alerter-2qd9t\" (UID: \"802d16e2-7d79-4df2-bf7b-304ab45e62ce\") " pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753101    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d549c5b-da45-458e-8451-e0627a418526-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753094    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-systemd\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753130    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/802d16e2-7d79-4df2-bf7b-304ab45e62ce-iptables-alerter-script\") pod \"iptables-alerter-2qd9t\" (UID: \"802d16e2-7d79-4df2-bf7b-304ab45e62ce\") " pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753151    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-sys-fs\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753162    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-systemd\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753169    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b63ae85-7c59-4a60-9f0e-862df0a82739-tmp\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753198    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753208    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d549c5b-da45-458e-8451-e0627a418526-cni-binary-copy\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753224    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bst84\" (UniqueName: \"kubernetes.io/projected/802d16e2-7d79-4df2-bf7b-304ab45e62ce-kube-api-access-bst84\") pod \"iptables-alerter-2qd9t\" (UID: \"802d16e2-7d79-4df2-bf7b-304ab45e62ce\") " pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753256    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pts4t\" (UniqueName: \"kubernetes.io/projected/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-kube-api-access-pts4t\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.753719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753227    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b7ec0a1-a68d-4699-bb5d-c3152791a0ef-tmp-dir\") pod \"node-resolver-mqn5j\" (UID: \"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef\") " pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753281    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/94cf40a2-35af-402c-b86d-f023b2ba9c85-agent-certs\") pod \"konnectivity-agent-hcnk6\" (UID: \"94cf40a2-35af-402c-b86d-f023b2ba9c85\") " pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753288    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-sysctl-conf\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753310    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d549c5b-da45-458e-8451-e0627a418526-system-cni-dir\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753355    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d549c5b-da45-458e-8451-e0627a418526-system-cni-dir\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753427    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2d549c5b-da45-458e-8451-e0627a418526-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753464    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-sys\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753524    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4v7x\" (UniqueName: \"kubernetes.io/projected/1b63ae85-7c59-4a60-9f0e-862df0a82739-kube-api-access-w4v7x\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753556    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-sys\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753580    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-host\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753610    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/94cf40a2-35af-402c-b86d-f023b2ba9c85-konnectivity-ca\") pod \"konnectivity-agent-hcnk6\" (UID: \"94cf40a2-35af-402c-b86d-f023b2ba9c85\") " pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753691    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ca49e661-dd33-4c7c-9751-408033360944-kubelet-config\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753726    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6b7ec0a1-a68d-4699-bb5d-c3152791a0ef-hosts-file\") pod \"node-resolver-mqn5j\" (UID: \"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef\") " pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753746    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-host\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753770    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzcxs\" (UniqueName: \"kubernetes.io/projected/6b7ec0a1-a68d-4699-bb5d-c3152791a0ef-kube-api-access-rzcxs\") pod \"node-resolver-mqn5j\" (UID: \"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef\") " pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753824    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f5cb386-9dde-4e3c-9122-eb1c6042e0e4-serviceca\") pod \"node-ca-k28bb\" (UID: \"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4\") " pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753808    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ca49e661-dd33-4c7c-9751-408033360944-kubelet-config\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.753903    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6b7ec0a1-a68d-4699-bb5d-c3152791a0ef-hosts-file\") pod \"node-resolver-mqn5j\" (UID: \"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef\") " pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.754571 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.753474    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:33.754037    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs podName:dc48a45a-1993-4cba-a9dc-636f75057239 nodeName:}" failed. No retries permitted until 2026-09-15 12:25:34.253987991 +0000 UTC m=+2.065899414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs") pod "network-metrics-daemon-rqsvh" (UID: "dc48a45a-1993-4cba-a9dc-636f75057239") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754063    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2d549c5b-da45-458e-8451-e0627a418526-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754063    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2d549c5b-da45-458e-8451-e0627a418526-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754122    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ca49e661-dd33-4c7c-9751-408033360944-dbus\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754165    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8drxh\" (UniqueName: \"kubernetes.io/projected/2d549c5b-da45-458e-8451-e0627a418526-kube-api-access-8drxh\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754193    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-sysconfig\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754233    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-var-lib-kubelet\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754259    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-etc-selinux\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754286    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-registration-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754313    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-tuned\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754340    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-device-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754400    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/94cf40a2-35af-402c-b86d-f023b2ba9c85-konnectivity-ca\") pod \"konnectivity-agent-hcnk6\" (UID: \"94cf40a2-35af-402c-b86d-f023b2ba9c85\") " pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754445    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-device-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754466    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-sysconfig\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754484    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ca49e661-dd33-4c7c-9751-408033360944-dbus\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:33.755564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754524    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-registration-dir\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.756311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754402    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f5cb386-9dde-4e3c-9122-eb1c6042e0e4-serviceca\") pod \"node-ca-k28bb\" (UID: \"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4\") " pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.756311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754527    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b63ae85-7c59-4a60-9f0e-862df0a82739-var-lib-kubelet\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.756311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754540    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2d549c5b-da45-458e-8451-e0627a418526-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.756311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.754540    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-etc-selinux\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.756311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.756169    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/94cf40a2-35af-402c-b86d-f023b2ba9c85-agent-certs\") pod \"konnectivity-agent-hcnk6\" (UID: \"94cf40a2-35af-402c-b86d-f023b2ba9c85\") " pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:33.756311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.756305    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b63ae85-7c59-4a60-9f0e-862df0a82739-tmp\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.756506 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.756490    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1b63ae85-7c59-4a60-9f0e-862df0a82739-etc-tuned\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.768360 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.768333    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p666k\" (UniqueName: \"kubernetes.io/projected/2f5cb386-9dde-4e3c-9122-eb1c6042e0e4-kube-api-access-p666k\") pod \"node-ca-k28bb\" (UID: \"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4\") " pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.768754 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.768713    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvbh5\" (UniqueName: \"kubernetes.io/projected/dc48a45a-1993-4cba-a9dc-636f75057239-kube-api-access-cvbh5\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:33.768905 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.768817    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pts4t\" (UniqueName: \"kubernetes.io/projected/dd0ef3c3-bb45-466d-a6f3-0b88472ef148-kube-api-access-pts4t\") pod \"aws-ebs-csi-driver-node-nnfv8\" (UID: \"dd0ef3c3-bb45-466d-a6f3-0b88472ef148\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.770315 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.770286    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bst84\" (UniqueName: \"kubernetes.io/projected/802d16e2-7d79-4df2-bf7b-304ab45e62ce-kube-api-access-bst84\") pod \"iptables-alerter-2qd9t\" (UID: \"802d16e2-7d79-4df2-bf7b-304ab45e62ce\") " pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.770909 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.770882    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8drxh\" (UniqueName: \"kubernetes.io/projected/2d549c5b-da45-458e-8451-e0627a418526-kube-api-access-8drxh\") pod \"multus-additional-cni-plugins-xl5dx\" (UID: \"2d549c5b-da45-458e-8451-e0627a418526\") " pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.771061 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.771043    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4v7x\" (UniqueName: \"kubernetes.io/projected/1b63ae85-7c59-4a60-9f0e-862df0a82739-kube-api-access-w4v7x\") pod \"tuned-zf9c6\" (UID: \"1b63ae85-7c59-4a60-9f0e-862df0a82739\") " pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.771104 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.771085    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzcxs\" (UniqueName: \"kubernetes.io/projected/6b7ec0a1-a68d-4699-bb5d-c3152791a0ef-kube-api-access-rzcxs\") pod \"node-resolver-mqn5j\" (UID: \"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef\") " pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.867217 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.867128    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p5wb9"
Sep 15 12:25:33.878834 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.878795    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86349040_e321_4bfa_b5fa_009b7c2a8cf5.slice/crio-02a69616343cf4dedc717444cb3cea9e9696291327ea29a7b622c5a4cb4cef54 WatchSource:0}: Error finding container 02a69616343cf4dedc717444cb3cea9e9696291327ea29a7b622c5a4cb4cef54: Status 404 returned error can't find the container with id 02a69616343cf4dedc717444cb3cea9e9696291327ea29a7b622c5a4cb4cef54
Sep 15 12:25:33.888356 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.888332    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:25:33.893594 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.893573    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zf9c6"
Sep 15 12:25:33.897359 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.897333    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b87a7f_98d6_4b99_928c_284f93a63928.slice/crio-ecb7f3c066d9307b7268b6a0e06dcd51b460a635f4be87374a8964a066e5c69a WatchSource:0}: Error finding container ecb7f3c066d9307b7268b6a0e06dcd51b460a635f4be87374a8964a066e5c69a: Status 404 returned error can't find the container with id ecb7f3c066d9307b7268b6a0e06dcd51b460a635f4be87374a8964a066e5c69a
Sep 15 12:25:33.902528 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.902503    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b63ae85_7c59_4a60_9f0e_862df0a82739.slice/crio-2add5f92dd89d39d1f40d51ea0857f576b83438431c3bc526b2d4e27d8066033 WatchSource:0}: Error finding container 2add5f92dd89d39d1f40d51ea0857f576b83438431c3bc526b2d4e27d8066033: Status 404 returned error can't find the container with id 2add5f92dd89d39d1f40d51ea0857f576b83438431c3bc526b2d4e27d8066033
Sep 15 12:25:33.908000 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.907981    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mqn5j"
Sep 15 12:25:33.918025 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.918002    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7ec0a1_a68d_4699_bb5d_c3152791a0ef.slice/crio-9dbccdfc4bf01668c6a550318a983af6d13e0b4cb3cd7aa6ba2bfbd78771da5c WatchSource:0}: Error finding container 9dbccdfc4bf01668c6a550318a983af6d13e0b4cb3cd7aa6ba2bfbd78771da5c: Status 404 returned error can't find the container with id 9dbccdfc4bf01668c6a550318a983af6d13e0b4cb3cd7aa6ba2bfbd78771da5c
Sep 15 12:25:33.923138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.923122    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k28bb"
Sep 15 12:25:33.932945 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.932917    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5cb386_9dde_4e3c_9122_eb1c6042e0e4.slice/crio-646b3090430cec870fac5986543d7724d18ec05846a4bb6e6bf91903f9e9fa35 WatchSource:0}: Error finding container 646b3090430cec870fac5986543d7724d18ec05846a4bb6e6bf91903f9e9fa35: Status 404 returned error can't find the container with id 646b3090430cec870fac5986543d7724d18ec05846a4bb6e6bf91903f9e9fa35
Sep 15 12:25:33.942188 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.942167    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xl5dx"
Sep 15 12:25:33.950966 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.950764    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d549c5b_da45_458e_8451_e0627a418526.slice/crio-ae82520fd88642632cbe9073761c97aca258bceab5557843ec230e14bc8e2a50 WatchSource:0}: Error finding container ae82520fd88642632cbe9073761c97aca258bceab5557843ec230e14bc8e2a50: Status 404 returned error can't find the container with id ae82520fd88642632cbe9073761c97aca258bceab5557843ec230e14bc8e2a50
Sep 15 12:25:33.957700 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.957679    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2qd9t"
Sep 15 12:25:33.967991 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.967832    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod802d16e2_7d79_4df2_bf7b_304ab45e62ce.slice/crio-edaec76ca1ee818d387e5662d9d3da9de947a49a0cf5657c10b0b6d4edca72cb WatchSource:0}: Error finding container edaec76ca1ee818d387e5662d9d3da9de947a49a0cf5657c10b0b6d4edca72cb: Status 404 returned error can't find the container with id edaec76ca1ee818d387e5662d9d3da9de947a49a0cf5657c10b0b6d4edca72cb
Sep 15 12:25:33.982704 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.982682    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:33.987528 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:33.987502    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8"
Sep 15 12:25:33.992219 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.992191    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94cf40a2_35af_402c_b86d_f023b2ba9c85.slice/crio-cca1239294ee750ffe24f024f50c81743bc006ceccf94dd18cc0eca6ffd27e10 WatchSource:0}: Error finding container cca1239294ee750ffe24f024f50c81743bc006ceccf94dd18cc0eca6ffd27e10: Status 404 returned error can't find the container with id cca1239294ee750ffe24f024f50c81743bc006ceccf94dd18cc0eca6ffd27e10
Sep 15 12:25:33.997653 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:25:33.997631    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0ef3c3_bb45_466d_a6f3_0b88472ef148.slice/crio-8cd12e402d63bd684f6cb23a9751ef6cbff56154b8d3f0899b642dd0b5f8fc16 WatchSource:0}: Error finding container 8cd12e402d63bd684f6cb23a9751ef6cbff56154b8d3f0899b642dd0b5f8fc16: Status 404 returned error can't find the container with id 8cd12e402d63bd684f6cb23a9751ef6cbff56154b8d3f0899b642dd0b5f8fc16
Sep 15 12:25:34.257119 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.257076    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:34.257326 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.257155    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7sq9\" (UniqueName: \"kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9\") pod \"network-check-target-z4d6q\" (UID: \"dbd143a3-56aa-4bef-b0a9-792ba54e447e\") " pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:34.257326 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.257191    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:34.257326 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:34.257262    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:34.257326 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:34.257320    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:34.257530 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:34.257347    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret podName:ca49e661-dd33-4c7c-9751-408033360944 nodeName:}" failed. No retries permitted until 2026-09-15 12:25:35.257325328 +0000 UTC m=+3.069236752 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret") pod "global-pull-secret-syncer-2n7pd" (UID: "ca49e661-dd33-4c7c-9751-408033360944") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:34.257530 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:34.257346    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:25:34.257530 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:34.257372    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:25:34.257530 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:34.257375    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs podName:dc48a45a-1993-4cba-a9dc-636f75057239 nodeName:}" failed. No retries permitted until 2026-09-15 12:25:35.257358328 +0000 UTC m=+3.069269765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs") pod "network-metrics-daemon-rqsvh" (UID: "dc48a45a-1993-4cba-a9dc-636f75057239") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:34.257530 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:34.257389    2586 projected.go:196] Error preparing data for projected volume kube-api-access-r7sq9 for pod openshift-network-diagnostics/network-check-target-z4d6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:34.257530 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:34.257445    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9 podName:dbd143a3-56aa-4bef-b0a9-792ba54e447e nodeName:}" failed. No retries permitted until 2026-09-15 12:25:35.257429461 +0000 UTC m=+3.069340897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7sq9" (UniqueName: "kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9") pod "network-check-target-z4d6q" (UID: "dbd143a3-56aa-4bef-b0a9-792ba54e447e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:34.458998 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.458960    2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:25:34.643013 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.642834    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 12:20:34 +0000 UTC" deadline="2028-04-17 01:10:41.621853795 +0000 UTC"
Sep 15 12:25:34.643013 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.642897    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13908h45m6.978962024s"
Sep 15 12:25:34.677611 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.677536    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal" event={"ID":"4baa257649766985809baa75fc95af2e","Type":"ContainerStarted","Data":"7241473a18726bdd07b161b0eb6797d0e30c19867ce112e45bfbab49d5f916ea"}
Sep 15 12:25:34.692304 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.692250    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal" event={"ID":"1d46ff4d7787a6b16b6c6c5774307130","Type":"ContainerStarted","Data":"4e8680af98d969b400a284a76907308c3a49a3bf150afb97b3d06bbd84878d26"}
Sep 15 12:25:34.708112 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.708061    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8" event={"ID":"dd0ef3c3-bb45-466d-a6f3-0b88472ef148","Type":"ContainerStarted","Data":"8cd12e402d63bd684f6cb23a9751ef6cbff56154b8d3f0899b642dd0b5f8fc16"}
Sep 15 12:25:34.720472 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.720428    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hcnk6" event={"ID":"94cf40a2-35af-402c-b86d-f023b2ba9c85","Type":"ContainerStarted","Data":"cca1239294ee750ffe24f024f50c81743bc006ceccf94dd18cc0eca6ffd27e10"}
Sep 15 12:25:34.734406 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.734363    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2qd9t" event={"ID":"802d16e2-7d79-4df2-bf7b-304ab45e62ce","Type":"ContainerStarted","Data":"edaec76ca1ee818d387e5662d9d3da9de947a49a0cf5657c10b0b6d4edca72cb"}
Sep 15 12:25:34.740473 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.740375    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl5dx" event={"ID":"2d549c5b-da45-458e-8451-e0627a418526","Type":"ContainerStarted","Data":"ae82520fd88642632cbe9073761c97aca258bceab5557843ec230e14bc8e2a50"}
Sep 15 12:25:34.754596 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.754554    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mqn5j" event={"ID":"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef","Type":"ContainerStarted","Data":"9dbccdfc4bf01668c6a550318a983af6d13e0b4cb3cd7aa6ba2bfbd78771da5c"}
Sep 15 12:25:34.757482 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.757448    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx" event={"ID":"21b87a7f-98d6-4b99-928c-284f93a63928","Type":"ContainerStarted","Data":"ecb7f3c066d9307b7268b6a0e06dcd51b460a635f4be87374a8964a066e5c69a"}
Sep 15 12:25:34.761144 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.761110    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p5wb9" event={"ID":"86349040-e321-4bfa-b5fa-009b7c2a8cf5","Type":"ContainerStarted","Data":"02a69616343cf4dedc717444cb3cea9e9696291327ea29a7b622c5a4cb4cef54"}
Sep 15 12:25:34.767711 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.767665    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k28bb" event={"ID":"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4","Type":"ContainerStarted","Data":"646b3090430cec870fac5986543d7724d18ec05846a4bb6e6bf91903f9e9fa35"}
Sep 15 12:25:34.771356 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:34.771322    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zf9c6" event={"ID":"1b63ae85-7c59-4a60-9f0e-862df0a82739","Type":"ContainerStarted","Data":"2add5f92dd89d39d1f40d51ea0857f576b83438431c3bc526b2d4e27d8066033"}
Sep 15 12:25:35.115636 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:35.112682    2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:25:35.264895 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:35.264850    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7sq9\" (UniqueName: \"kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9\") pod \"network-check-target-z4d6q\" (UID: \"dbd143a3-56aa-4bef-b0a9-792ba54e447e\") " pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:35.265117 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:35.264913    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:35.265117 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:35.264979    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:35.265236 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.265192    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:35.265289 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.265262    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret podName:ca49e661-dd33-4c7c-9751-408033360944 nodeName:}" failed. No retries permitted until 2026-09-15 12:25:37.265242434 +0000 UTC m=+5.077153873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret") pod "global-pull-secret-syncer-2n7pd" (UID: "ca49e661-dd33-4c7c-9751-408033360944") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:35.265769 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.265746    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:25:35.265885 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.265778    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:25:35.265885 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.265792    2586 projected.go:196] Error preparing data for projected volume kube-api-access-r7sq9 for pod openshift-network-diagnostics/network-check-target-z4d6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:35.265885 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.265846    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9 podName:dbd143a3-56aa-4bef-b0a9-792ba54e447e nodeName:}" failed. No retries permitted until 2026-09-15 12:25:37.265828771 +0000 UTC m=+5.077740196 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7sq9" (UniqueName: "kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9") pod "network-check-target-z4d6q" (UID: "dbd143a3-56aa-4bef-b0a9-792ba54e447e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:35.266068 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.265930    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:35.266068 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.265968    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs podName:dc48a45a-1993-4cba-a9dc-636f75057239 nodeName:}" failed. No retries permitted until 2026-09-15 12:25:37.265955467 +0000 UTC m=+5.077866891 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs") pod "network-metrics-daemon-rqsvh" (UID: "dc48a45a-1993-4cba-a9dc-636f75057239") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:35.644049 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:35.643979    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 12:20:34 +0000 UTC" deadline="2028-02-08 19:37:25.421667556 +0000 UTC"
Sep 15 12:25:35.644049 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:35.644045    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12271h11m49.777626681s"
Sep 15 12:25:35.646252 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:35.646226    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:35.646395 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.646355    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:35.647041 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:35.647020    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:35.647164 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.647142    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:35.647229 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:35.647220    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:35.647327 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:35.647310    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:35.880137 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:35.880099    2586 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:25:36.163061 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:36.163027    2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:25:37.283872 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:37.283313    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7sq9\" (UniqueName: \"kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9\") pod \"network-check-target-z4d6q\" (UID: \"dbd143a3-56aa-4bef-b0a9-792ba54e447e\") " pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:37.283872 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:37.283374    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:37.283872 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:37.283438    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:37.283872 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.283599    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:37.283872 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.283669    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret podName:ca49e661-dd33-4c7c-9751-408033360944 nodeName:}" failed. No retries permitted until 2026-09-15 12:25:41.283648343 +0000 UTC m=+9.095559770 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret") pod "global-pull-secret-syncer-2n7pd" (UID: "ca49e661-dd33-4c7c-9751-408033360944") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:37.284567 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.284144    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:25:37.284567 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.284165    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:25:37.284567 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.284179    2586 projected.go:196] Error preparing data for projected volume kube-api-access-r7sq9 for pod openshift-network-diagnostics/network-check-target-z4d6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:37.284567 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.284230    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9 podName:dbd143a3-56aa-4bef-b0a9-792ba54e447e nodeName:}" failed. No retries permitted until 2026-09-15 12:25:41.284211479 +0000 UTC m=+9.096122902 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7sq9" (UniqueName: "kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9") pod "network-check-target-z4d6q" (UID: "dbd143a3-56aa-4bef-b0a9-792ba54e447e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:37.284567 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.284311    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:37.284567 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.284353    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs podName:dc48a45a-1993-4cba-a9dc-636f75057239 nodeName:}" failed. No retries permitted until 2026-09-15 12:25:41.284339623 +0000 UTC m=+9.096251048 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs") pod "network-metrics-daemon-rqsvh" (UID: "dc48a45a-1993-4cba-a9dc-636f75057239") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:37.646605 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:37.646513    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:37.646794 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.646663    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:37.647182 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:37.647160    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:37.647306 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.647270    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:37.647376 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:37.647323    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:37.647433 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:37.647405    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:39.647433 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:39.646754    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:39.647433 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:39.646775    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:39.647433 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:39.646934    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:39.647433 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:39.646949    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:39.647433 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:39.647068    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:39.647433 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:39.647158    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:41.320987 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:41.320930    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7sq9\" (UniqueName: \"kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9\") pod \"network-check-target-z4d6q\" (UID: \"dbd143a3-56aa-4bef-b0a9-792ba54e447e\") " pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:41.320987 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:41.320996    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:41.321537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:41.321059    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:41.321537 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.321199    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:41.321537 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.321204    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:25:41.321537 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.321238    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:25:41.321537 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.321253    2586 projected.go:196] Error preparing data for projected volume kube-api-access-r7sq9 for pod openshift-network-diagnostics/network-check-target-z4d6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:41.321537 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.321274    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret podName:ca49e661-dd33-4c7c-9751-408033360944 nodeName:}" failed. No retries permitted until 2026-09-15 12:25:49.321252679 +0000 UTC m=+17.133164121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret") pod "global-pull-secret-syncer-2n7pd" (UID: "ca49e661-dd33-4c7c-9751-408033360944") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:41.321537 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.321280    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:41.321537 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.321311    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9 podName:dbd143a3-56aa-4bef-b0a9-792ba54e447e nodeName:}" failed. No retries permitted until 2026-09-15 12:25:49.321293446 +0000 UTC m=+17.133204880 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7sq9" (UniqueName: "kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9") pod "network-check-target-z4d6q" (UID: "dbd143a3-56aa-4bef-b0a9-792ba54e447e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:41.321537 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.321338    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs podName:dc48a45a-1993-4cba-a9dc-636f75057239 nodeName:}" failed. No retries permitted until 2026-09-15 12:25:49.321321403 +0000 UTC m=+17.133232842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs") pod "network-metrics-daemon-rqsvh" (UID: "dc48a45a-1993-4cba-a9dc-636f75057239") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:41.646441 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:41.646345    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:41.646441 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:41.646378    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:41.646693 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:41.646357    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:41.646693 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.646520    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:41.646693 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.646671    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:41.646882 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:41.646777    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:43.646232 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:43.646161    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:43.646873 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:43.646360    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:43.646873 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:43.646827    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:43.647008 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:43.646834    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:43.647008 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:43.646964    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:43.647109 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:43.647067    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:45.646634 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:45.646560    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:45.646634 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:45.646589    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:45.646634 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:45.646596    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:45.647151 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:45.646695    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:45.647151 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:45.646741    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:45.647151 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:45.646847    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:47.646978 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:47.646940    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:47.646978 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:47.646971    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:47.647615 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:47.646940    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:47.647615 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:47.647065    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:47.647615 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:47.647210    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:47.647615 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:47.647316    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:49.383400 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:49.383357    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7sq9\" (UniqueName: \"kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9\") pod \"network-check-target-z4d6q\" (UID: \"dbd143a3-56aa-4bef-b0a9-792ba54e447e\") " pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:49.383400 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:49.383404    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:49.383902 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:49.383444    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:49.383902 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.383538    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:49.383902 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.383553    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:25:49.383902 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.383576    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:49.383902 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.383608    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret podName:ca49e661-dd33-4c7c-9751-408033360944 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:05.383589196 +0000 UTC m=+33.195500639 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret") pod "global-pull-secret-syncer-2n7pd" (UID: "ca49e661-dd33-4c7c-9751-408033360944") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:25:49.383902 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.383631    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs podName:dc48a45a-1993-4cba-a9dc-636f75057239 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:05.383614838 +0000 UTC m=+33.195526262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs") pod "network-metrics-daemon-rqsvh" (UID: "dc48a45a-1993-4cba-a9dc-636f75057239") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:25:49.383902 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.383584    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:25:49.383902 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.383653    2586 projected.go:196] Error preparing data for projected volume kube-api-access-r7sq9 for pod openshift-network-diagnostics/network-check-target-z4d6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:49.383902 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.383691    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9 podName:dbd143a3-56aa-4bef-b0a9-792ba54e447e nodeName:}" failed. No retries permitted until 2026-09-15 12:26:05.383680492 +0000 UTC m=+33.195591912 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7sq9" (UniqueName: "kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9") pod "network-check-target-z4d6q" (UID: "dbd143a3-56aa-4bef-b0a9-792ba54e447e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:25:49.646838 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:49.646748    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:49.646989 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:49.646754    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:49.646989 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:49.646890    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:49.646989 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.646898    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:49.647102 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.646982    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:49.647102 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:49.647056    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:51.646899 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:51.646865    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:51.647369 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:51.646866    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:51.647369 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:51.646984    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:51.647369 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:51.647087    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:51.647369 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:51.646866    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:51.647369 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:51.647175    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:53.646644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:53.646571    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:53.647085 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:53.646694    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:53.647085 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:53.646703    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:53.647085 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:53.646698    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:53.647085 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:53.646806    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:53.647085 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:53.646873    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:54.819947 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.819702    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p5wb9" event={"ID":"86349040-e321-4bfa-b5fa-009b7c2a8cf5","Type":"ContainerStarted","Data":"0a056283e338a2c9391bc9592f7295dfbff3115c6e116eac267566405b57bd93"}
Sep 15 12:25:54.821237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.821213    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k28bb" event={"ID":"2f5cb386-9dde-4e3c-9122-eb1c6042e0e4","Type":"ContainerStarted","Data":"caa41e85ef75f0371bf0ce00be286835bcbdf9f68fbc8ffcf2e4324d566681b9"}
Sep 15 12:25:54.822616 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.822593    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zf9c6" event={"ID":"1b63ae85-7c59-4a60-9f0e-862df0a82739","Type":"ContainerStarted","Data":"ce3bf2803997d3522a37df4343e1e21e32a97bae7a99d4e7c48db58b75ce3330"}
Sep 15 12:25:54.824076 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.824052    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal" event={"ID":"4baa257649766985809baa75fc95af2e","Type":"ContainerStarted","Data":"36f3a71687c9d1f8af15e047f2067b6b6afe922ea832c6a9ad14889f33c001db"}
Sep 15 12:25:54.825446 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.825422    2586 generic.go:356] "Generic (PLEG): container finished" podID="1d46ff4d7787a6b16b6c6c5774307130" containerID="9c1e6017ce82dc9bb0594575aa632373d0ee2140b170767145cf9cca36f7653c" exitCode=0
Sep 15 12:25:54.825534 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.825489    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal" event={"ID":"1d46ff4d7787a6b16b6c6c5774307130","Type":"ContainerDied","Data":"9c1e6017ce82dc9bb0594575aa632373d0ee2140b170767145cf9cca36f7653c"}
Sep 15 12:25:54.827111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.827090    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8" event={"ID":"dd0ef3c3-bb45-466d-a6f3-0b88472ef148","Type":"ContainerStarted","Data":"4c94a4ad61fdde27523141f19c015a6623592f94c26801d2f8d3aabba911c5ea"}
Sep 15 12:25:54.828545 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.828527    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hcnk6" event={"ID":"94cf40a2-35af-402c-b86d-f023b2ba9c85","Type":"ContainerStarted","Data":"e7ec5f518200dba235e049cdde72d2986df18069a0cad3193dbc9873baa01944"}
Sep 15 12:25:54.830123 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.830103    2586 generic.go:356] "Generic (PLEG): container finished" podID="2d549c5b-da45-458e-8451-e0627a418526" containerID="6a5a6b40fb40c62222d7673d4e1ce18b395a7980c0805aa346faaea9be9a662d" exitCode=0
Sep 15 12:25:54.830208 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.830145    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl5dx" event={"ID":"2d549c5b-da45-458e-8451-e0627a418526","Type":"ContainerDied","Data":"6a5a6b40fb40c62222d7673d4e1ce18b395a7980c0805aa346faaea9be9a662d"}
Sep 15 12:25:54.831584 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.831572    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mqn5j" event={"ID":"6b7ec0a1-a68d-4699-bb5d-c3152791a0ef","Type":"ContainerStarted","Data":"8a608a81379eb5f147377b9c276cf092ac333b5c1df9f5937d0d764de3340f50"}
Sep 15 12:25:54.834926 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.834906    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx" event={"ID":"21b87a7f-98d6-4b99-928c-284f93a63928","Type":"ContainerStarted","Data":"f018d180128d54beaaa569258107af9acb75abe790bdceea5d5879ed952ebf8e"}
Sep 15 12:25:54.835027 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.834977    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx" event={"ID":"21b87a7f-98d6-4b99-928c-284f93a63928","Type":"ContainerStarted","Data":"43963ae133ad99ff83ed3f02831f5b1f16d2caa649a55f74924deb720d790104"}
Sep 15 12:25:54.835090 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.835033    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx" event={"ID":"21b87a7f-98d6-4b99-928c-284f93a63928","Type":"ContainerStarted","Data":"d22558c599436efae83b12e502161ca71d8e45708d3f9c3c51eee02318b070fd"}
Sep 15 12:25:54.835090 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.835085    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx" event={"ID":"21b87a7f-98d6-4b99-928c-284f93a63928","Type":"ContainerStarted","Data":"eed2b6fbc8510bd32f74b9949c27c3560d6a0a4b2e8341e906071648782b7f1d"}
Sep 15 12:25:54.835192 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.835137    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx" event={"ID":"21b87a7f-98d6-4b99-928c-284f93a63928","Type":"ContainerStarted","Data":"cb7b3f20d439abf882b2496caf833146923c76d678429aa3e4887d208858c358"}
Sep 15 12:25:54.835244 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.835184    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx" event={"ID":"21b87a7f-98d6-4b99-928c-284f93a63928","Type":"ContainerStarted","Data":"41a7c598f7dd728827dd9ab22f30fb0ab4d9bef8708db54739b45e35d71f607b"}
Sep 15 12:25:54.836253 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.836210    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-p5wb9" podStartSLOduration=1.835802359 podStartE2EDuration="21.836199173s" totalImagesPullingTime="20.000396814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:25:54.836020373 +0000 UTC m=+22.647931814" watchObservedRunningTime="2026-09-15 12:25:54.836199173 +0000 UTC m=+22.648110646"
Sep 15 12:25:54.850056 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.850012    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k28bb" podStartSLOduration=2.132765496 podStartE2EDuration="21.849996089s" totalImagesPullingTime="19.717230593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:25:54.84980901 +0000 UTC m=+22.661720453" watchObservedRunningTime="2026-09-15 12:25:54.849996089 +0000 UTC m=+22.661907532"
Sep 15 12:25:54.877879 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.877843    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mqn5j" podStartSLOduration=2.145838709 podStartE2EDuration="21.877829255s" totalImagesPullingTime="19.731990546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:25:54.877548067 +0000 UTC m=+22.689459509" watchObservedRunningTime="2026-09-15 12:25:54.877829255 +0000 UTC m=+22.689740696"
Sep 15 12:25:54.914345 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.914309    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hcnk6" podStartSLOduration=10.491868282 podStartE2EDuration="21.914295121s" totalImagesPullingTime="11.422426839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:25:54.914234087 +0000 UTC m=+22.726145530" watchObservedRunningTime="2026-09-15 12:25:54.914295121 +0000 UTC m=+22.726206563"
Sep 15 12:25:54.931586 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.931505    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zf9c6" podStartSLOduration=2.184047272 podStartE2EDuration="21.931490843s" totalImagesPullingTime="19.747443571s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:25:54.9314253 +0000 UTC m=+22.743336742" watchObservedRunningTime="2026-09-15 12:25:54.931490843 +0000 UTC m=+22.743402284"
Sep 15 12:25:54.947169 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:54.947122    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-20.ec2.internal" podStartSLOduration=20.947107741 podStartE2EDuration="20.947107741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:25:54.946759549 +0000 UTC m=+22.758670989" watchObservedRunningTime="2026-09-15 12:25:54.947107741 +0000 UTC m=+22.759019184"
Sep 15 12:25:55.646237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:55.646197    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:55.646237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:55.646225    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:55.646488 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:55.646225    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:55.646488 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:55.646335    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:55.646488 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:55.646397    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:55.646488 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:55.646475    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:55.839496 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:55.839461    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2qd9t" event={"ID":"802d16e2-7d79-4df2-bf7b-304ab45e62ce","Type":"ContainerStarted","Data":"f77a52cb1af53e52d7a112c1f6915cc96b45944e11e92b305fa022867d23da75"}
Sep 15 12:25:55.841487 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:55.841453    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal" event={"ID":"1d46ff4d7787a6b16b6c6c5774307130","Type":"ContainerStarted","Data":"f5fdd18433d3a2f29f7c53f8aeb11e14e1cacf2eac3a8dd774677cb1afe69371"}
Sep 15 12:25:55.853605 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:55.853557    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2qd9t" podStartSLOduration=3.171584343 podStartE2EDuration="22.85354269s" totalImagesPullingTime="19.681958347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:25:55.853302795 +0000 UTC m=+23.665214237" watchObservedRunningTime="2026-09-15 12:25:55.85354269 +0000 UTC m=+23.665454132"
Sep 15 12:25:55.868326 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:55.868280    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-20.ec2.internal" podStartSLOduration=21.868267308 podStartE2EDuration="21.868267308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:25:55.867518766 +0000 UTC m=+23.679430208" watchObservedRunningTime="2026-09-15 12:25:55.868267308 +0000 UTC m=+23.680178751"
Sep 15 12:25:55.896876 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:55.896798    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:55.897656 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:55.897637    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:56.847654 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:56.847565    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx" event={"ID":"21b87a7f-98d6-4b99-928c-284f93a63928","Type":"ContainerStarted","Data":"9ea832e52bc5285d233e881aab7a3db1e96e1694a39d23b6f3335aa66195c0bb"}
Sep 15 12:25:57.646645 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:57.646466    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:57.646817 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:57.646466    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:57.646817 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:57.646750    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:57.646817 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:57.646466    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:57.646955 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:57.646851    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:57.646955 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:57.646882    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:57.849674 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:57.849644    2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 15 12:25:58.986847 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:58.986815    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:58.987231 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:58.986924    2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 15 12:25:58.987337 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:58.987320    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hcnk6"
Sep 15 12:25:59.412620 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.412447    2586 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 12:25:59.627870 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.627731    2586 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T12:25:59.41262136Z","UUID":"5b63940d-992a-4dca-9af8-715495a03406","Handler":null,"Name":"","Endpoint":""}
Sep 15 12:25:59.628961 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.628942    2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 12:25:59.629059 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.628969    2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 12:25:59.646065 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.646044    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:25:59.646164 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.646133    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:25:59.646164 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:59.646148    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:25:59.646269 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:59.646208    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:25:59.646269 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.646225    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:25:59.646354 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:25:59.646316    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:25:59.857894 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.857855    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8" event={"ID":"dd0ef3c3-bb45-466d-a6f3-0b88472ef148","Type":"ContainerStarted","Data":"b2e410fe2547807f2d88a5ac719ef5d5608b207b674730f1cd5f94a8578038c2"}
Sep 15 12:25:59.859640 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.859613    2586 generic.go:356] "Generic (PLEG): container finished" podID="2d549c5b-da45-458e-8451-e0627a418526" containerID="1372efb65d4353b0640486dc9392379f247c0cf3239340c76ea0fc16922cd3fc" exitCode=0
Sep 15 12:25:59.859766 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.859699    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl5dx" event={"ID":"2d549c5b-da45-458e-8451-e0627a418526","Type":"ContainerDied","Data":"1372efb65d4353b0640486dc9392379f247c0cf3239340c76ea0fc16922cd3fc"}
Sep 15 12:25:59.863295 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.863273    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx" event={"ID":"21b87a7f-98d6-4b99-928c-284f93a63928","Type":"ContainerStarted","Data":"56ea4be7497009d969b82671378291d6d78873fede4ae38c5be40a3eef4f0378"}
Sep 15 12:25:59.902571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:25:59.902487    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx" podStartSLOduration=7.122560959 podStartE2EDuration="26.902472275s" totalImagesPullingTime="19.779911316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:25:59.901962313 +0000 UTC m=+27.713873756" watchObservedRunningTime="2026-09-15 12:25:59.902472275 +0000 UTC m=+27.714383761"
Sep 15 12:26:00.867855 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:00.867819    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8" event={"ID":"dd0ef3c3-bb45-466d-a6f3-0b88472ef148","Type":"ContainerStarted","Data":"6ee5318064309ed6fe4d1481dd8e5a37ff6100ca1be67839cc03ddf2a1112c2c"}
Sep 15 12:26:00.870083 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:00.870055    2586 generic.go:356] "Generic (PLEG): container finished" podID="2d549c5b-da45-458e-8451-e0627a418526" containerID="cbabfa5fc3c8fc6749da77d203c0c22168e451ef1403ed3c66504579801b51e4" exitCode=0
Sep 15 12:26:00.870215 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:00.870146    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl5dx" event={"ID":"2d549c5b-da45-458e-8451-e0627a418526","Type":"ContainerDied","Data":"cbabfa5fc3c8fc6749da77d203c0c22168e451ef1403ed3c66504579801b51e4"}
Sep 15 12:26:00.870215 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:00.870192    2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 15 12:26:00.870644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:00.870618    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:26:00.870791 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:00.870693    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:26:00.885597 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:00.885540    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnfv8" podStartSLOduration=2.308946196 podStartE2EDuration="27.885523913s" totalImagesPullingTime="25.576577717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:00.88516365 +0000 UTC m=+28.697075093" watchObservedRunningTime="2026-09-15 12:26:00.885523913 +0000 UTC m=+28.697435388"
Sep 15 12:26:00.887921 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:00.887879    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:26:00.890149 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:00.888068    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:26:01.109968 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:01.109934    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:26:01.198511 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:01.198285    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rqsvh"]
Sep 15 12:26:01.198668 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:01.198616    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:26:01.198738 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:01.198713    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:26:01.201792 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:01.201746    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z4d6q"]
Sep 15 12:26:01.201919 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:01.201873    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:26:01.201993 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:01.201971    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:26:01.202448 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:01.202422    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2n7pd"]
Sep 15 12:26:01.202638 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:01.202535    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:26:01.202696 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:01.202632    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:26:02.646709 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:02.646667    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:26:02.647244 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:02.646744    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:26:02.647244 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:02.646823    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:26:02.647244 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:02.646950    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:26:02.647244 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:02.646998    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:26:02.647244 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:02.647090    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:26:02.877476 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:02.877439    2586 generic.go:356] "Generic (PLEG): container finished" podID="2d549c5b-da45-458e-8451-e0627a418526" containerID="48772fa04968afb17b19e196c7a51a5953b6371dc5a5e0c38709542d7bfac306" exitCode=0
Sep 15 12:26:02.877623 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:02.877516    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl5dx" event={"ID":"2d549c5b-da45-458e-8451-e0627a418526","Type":"ContainerDied","Data":"48772fa04968afb17b19e196c7a51a5953b6371dc5a5e0c38709542d7bfac306"}
Sep 15 12:26:04.646540 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:04.646508    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:26:04.647070 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:04.646508    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:26:04.647070 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:04.646643    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:26:04.647070 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:04.646640    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n7pd" podUID="ca49e661-dd33-4c7c-9751-408033360944"
Sep 15 12:26:04.647070 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:04.646730    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4d6q" podUID="dbd143a3-56aa-4bef-b0a9-792ba54e447e"
Sep 15 12:26:04.647070 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:04.646851    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqsvh" podUID="dc48a45a-1993-4cba-a9dc-636f75057239"
Sep 15 12:26:05.393991 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:05.393950    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:26:05.394171 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:05.394010    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7sq9\" (UniqueName: \"kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9\") pod \"network-check-target-z4d6q\" (UID: \"dbd143a3-56aa-4bef-b0a9-792ba54e447e\") " pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:26:05.394171 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:05.394039    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:26:05.394171 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:05.394113    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:26:05.394285 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:05.394177    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret podName:ca49e661-dd33-4c7c-9751-408033360944 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:37.394162499 +0000 UTC m=+65.206073923 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret") pod "global-pull-secret-syncer-2n7pd" (UID: "ca49e661-dd33-4c7c-9751-408033360944") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:26:05.394285 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:05.394184    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:26:05.394285 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:05.394113    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:26:05.394285 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:05.394209    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:26:05.394285 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:05.394225    2586 projected.go:196] Error preparing data for projected volume kube-api-access-r7sq9 for pod openshift-network-diagnostics/network-check-target-z4d6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:26:05.394285 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:05.394244    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs podName:dc48a45a-1993-4cba-a9dc-636f75057239 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:37.394231001 +0000 UTC m=+65.206142433 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs") pod "network-metrics-daemon-rqsvh" (UID: "dc48a45a-1993-4cba-a9dc-636f75057239") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:26:05.394285 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:05.394271    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9 podName:dbd143a3-56aa-4bef-b0a9-792ba54e447e nodeName:}" failed. No retries permitted until 2026-09-15 12:26:37.394258774 +0000 UTC m=+65.206170194 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7sq9" (UniqueName: "kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9") pod "network-check-target-z4d6q" (UID: "dbd143a3-56aa-4bef-b0a9-792ba54e447e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:26:06.454530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.454497    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-20.ec2.internal" event="NodeReady"
Sep 15 12:26:06.455036 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.454635    2586 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 12:26:06.496332 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.496295    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v8q24"]
Sep 15 12:26:06.522668 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.522641    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tp5lb"]
Sep 15 12:26:06.522876 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.522787    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.525321 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.525285    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 12:26:06.525321 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.525317    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 12:26:06.525548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.525520    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pzg8w\""
Sep 15 12:26:06.536815 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.536773    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v8q24"]
Sep 15 12:26:06.536938 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.536844    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tp5lb"]
Sep 15 12:26:06.536938 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.536880    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tp5lb"
Sep 15 12:26:06.539837 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.539812    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v4v6p\""
Sep 15 12:26:06.539983 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.539868    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 12:26:06.539983 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.539901    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 12:26:06.539983 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.539907    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 12:26:06.602666 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.602592    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4zjs\" (UniqueName: \"kubernetes.io/projected/040b0ab2-3bbc-4355-acf8-b78f7446bdf2-kube-api-access-f4zjs\") pod \"ingress-canary-tp5lb\" (UID: \"040b0ab2-3bbc-4355-acf8-b78f7446bdf2\") " pod="openshift-ingress-canary/ingress-canary-tp5lb"
Sep 15 12:26:06.602666 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.602628    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.602666 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.602645    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfhnq\" (UniqueName: \"kubernetes.io/projected/d76f9797-13c3-49fa-b018-65de900b8bb3-kube-api-access-tfhnq\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.602990 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.602731    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/040b0ab2-3bbc-4355-acf8-b78f7446bdf2-cert\") pod \"ingress-canary-tp5lb\" (UID: \"040b0ab2-3bbc-4355-acf8-b78f7446bdf2\") " pod="openshift-ingress-canary/ingress-canary-tp5lb"
Sep 15 12:26:06.602990 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.602834    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d76f9797-13c3-49fa-b018-65de900b8bb3-config-volume\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.602990 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.602907    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d76f9797-13c3-49fa-b018-65de900b8bb3-tmp-dir\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.646926 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.646893    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:26:06.647108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.646940    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:26:06.647108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.646897    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:26:06.649566 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.649541    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 12:26:06.649881 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.649858    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 12:26:06.649980 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.649896    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mnpg9\""
Sep 15 12:26:06.650041 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.649983    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 12:26:06.650041 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.650022    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 12:26:06.650138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.650109    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j4plp\""
Sep 15 12:26:06.703511 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.703484    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d76f9797-13c3-49fa-b018-65de900b8bb3-config-volume\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.703640 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.703565    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d76f9797-13c3-49fa-b018-65de900b8bb3-tmp-dir\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.703640 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.703608    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4zjs\" (UniqueName: \"kubernetes.io/projected/040b0ab2-3bbc-4355-acf8-b78f7446bdf2-kube-api-access-f4zjs\") pod \"ingress-canary-tp5lb\" (UID: \"040b0ab2-3bbc-4355-acf8-b78f7446bdf2\") " pod="openshift-ingress-canary/ingress-canary-tp5lb"
Sep 15 12:26:06.703640 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.703630    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.703813 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.703655    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfhnq\" (UniqueName: \"kubernetes.io/projected/d76f9797-13c3-49fa-b018-65de900b8bb3-kube-api-access-tfhnq\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.703813 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.703707    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/040b0ab2-3bbc-4355-acf8-b78f7446bdf2-cert\") pod \"ingress-canary-tp5lb\" (UID: \"040b0ab2-3bbc-4355-acf8-b78f7446bdf2\") " pod="openshift-ingress-canary/ingress-canary-tp5lb"
Sep 15 12:26:06.703943 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.703921    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d76f9797-13c3-49fa-b018-65de900b8bb3-tmp-dir\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.704044 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:06.704028    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:26:06.704091 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.704071    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d76f9797-13c3-49fa-b018-65de900b8bb3-config-volume\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.704134 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:06.704097    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls podName:d76f9797-13c3-49fa-b018-65de900b8bb3 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:07.204077854 +0000 UTC m=+35.015989288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls") pod "dns-default-v8q24" (UID: "d76f9797-13c3-49fa-b018-65de900b8bb3") : secret "dns-default-metrics-tls" not found
Sep 15 12:26:06.708561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.708541    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/040b0ab2-3bbc-4355-acf8-b78f7446bdf2-cert\") pod \"ingress-canary-tp5lb\" (UID: \"040b0ab2-3bbc-4355-acf8-b78f7446bdf2\") " pod="openshift-ingress-canary/ingress-canary-tp5lb"
Sep 15 12:26:06.712151 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.712125    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfhnq\" (UniqueName: \"kubernetes.io/projected/d76f9797-13c3-49fa-b018-65de900b8bb3-kube-api-access-tfhnq\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:06.712363 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.712304    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4zjs\" (UniqueName: \"kubernetes.io/projected/040b0ab2-3bbc-4355-acf8-b78f7446bdf2-kube-api-access-f4zjs\") pod \"ingress-canary-tp5lb\" (UID: \"040b0ab2-3bbc-4355-acf8-b78f7446bdf2\") " pod="openshift-ingress-canary/ingress-canary-tp5lb"
Sep 15 12:26:06.846481 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:06.846442    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tp5lb"
Sep 15 12:26:07.208091 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:07.207856    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:07.208091 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:07.208006    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:26:07.208352 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:07.208153    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls podName:d76f9797-13c3-49fa-b018-65de900b8bb3 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:08.208133782 +0000 UTC m=+36.020045201 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls") pod "dns-default-v8q24" (UID: "d76f9797-13c3-49fa-b018-65de900b8bb3") : secret "dns-default-metrics-tls" not found
Sep 15 12:26:07.250334 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:07.250304    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tp5lb"]
Sep 15 12:26:07.255439 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:07.255411    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod040b0ab2_3bbc_4355_acf8_b78f7446bdf2.slice/crio-a035a421b90d56b07f7836f8dbe3f47e4f6d63581e4d77b44429ff68eb0a4708 WatchSource:0}: Error finding container a035a421b90d56b07f7836f8dbe3f47e4f6d63581e4d77b44429ff68eb0a4708: Status 404 returned error can't find the container with id a035a421b90d56b07f7836f8dbe3f47e4f6d63581e4d77b44429ff68eb0a4708
Sep 15 12:26:07.892307 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:07.892274    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tp5lb" event={"ID":"040b0ab2-3bbc-4355-acf8-b78f7446bdf2","Type":"ContainerStarted","Data":"a035a421b90d56b07f7836f8dbe3f47e4f6d63581e4d77b44429ff68eb0a4708"}
Sep 15 12:26:07.895076 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:07.895051    2586 generic.go:356] "Generic (PLEG): container finished" podID="2d549c5b-da45-458e-8451-e0627a418526" containerID="5d35921d28a3ad03fbc5bc8f722ecb228807b79ba4365e0bcea6b51c8d1c344a" exitCode=0
Sep 15 12:26:07.895209 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:07.895096    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl5dx" event={"ID":"2d549c5b-da45-458e-8451-e0627a418526","Type":"ContainerDied","Data":"5d35921d28a3ad03fbc5bc8f722ecb228807b79ba4365e0bcea6b51c8d1c344a"}
Sep 15 12:26:08.216404 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:08.216267    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:08.216622 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:08.216405    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:26:08.216622 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:08.216484    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls podName:d76f9797-13c3-49fa-b018-65de900b8bb3 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:10.216462628 +0000 UTC m=+38.028374053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls") pod "dns-default-v8q24" (UID: "d76f9797-13c3-49fa-b018-65de900b8bb3") : secret "dns-default-metrics-tls" not found
Sep 15 12:26:08.901929 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:08.901887    2586 generic.go:356] "Generic (PLEG): container finished" podID="2d549c5b-da45-458e-8451-e0627a418526" containerID="5190e5f9d415dca5d015e3f0510ca53dbb720fcab304027d7124771660bf2b75" exitCode=0
Sep 15 12:26:08.902598 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:08.901963    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl5dx" event={"ID":"2d549c5b-da45-458e-8451-e0627a418526","Type":"ContainerDied","Data":"5190e5f9d415dca5d015e3f0510ca53dbb720fcab304027d7124771660bf2b75"}
Sep 15 12:26:09.908268 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:09.908227    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl5dx" event={"ID":"2d549c5b-da45-458e-8451-e0627a418526","Type":"ContainerStarted","Data":"85aa76a98d684425d6a23091368afdc4e01e1fc342294c0e9221e4dc95f32010"}
Sep 15 12:26:09.929413 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:09.929364    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xl5dx" podStartSLOduration=6.606883718 podStartE2EDuration="36.929341666s" totalImagesPullingTime="30.322457948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:09.928281482 +0000 UTC m=+37.740192923" watchObservedRunningTime="2026-09-15 12:26:09.929341666 +0000 UTC m=+37.741253101"
Sep 15 12:26:10.232959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:10.232928    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:10.233120 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:10.233067    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:26:10.233160 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:10.233131    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls podName:d76f9797-13c3-49fa-b018-65de900b8bb3 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:14.233116278 +0000 UTC m=+42.045027703 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls") pod "dns-default-v8q24" (UID: "d76f9797-13c3-49fa-b018-65de900b8bb3") : secret "dns-default-metrics-tls" not found
Sep 15 12:26:10.912268 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:10.912228    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tp5lb" event={"ID":"040b0ab2-3bbc-4355-acf8-b78f7446bdf2","Type":"ContainerStarted","Data":"bea7908e60f07c22233557eb230ab2cf0a638f34c93df573ddc4d57da4b173c1"}
Sep 15 12:26:10.929166 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:10.929123    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tp5lb" podStartSLOduration=2.224058695 podStartE2EDuration="4.929107096s" totalImagesPullingTime="2.705048401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:10.928637191 +0000 UTC m=+38.740548634" watchObservedRunningTime="2026-09-15 12:26:10.929107096 +0000 UTC m=+38.741018537"
Sep 15 12:26:14.261192 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:14.261148    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:14.261618 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:14.261312    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:26:14.261618 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:14.261379    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls podName:d76f9797-13c3-49fa-b018-65de900b8bb3 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:22.261359621 +0000 UTC m=+50.073271061 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls") pod "dns-default-v8q24" (UID: "d76f9797-13c3-49fa-b018-65de900b8bb3") : secret "dns-default-metrics-tls" not found
Sep 15 12:26:22.315800 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:22.315744    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:22.319067 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:22.319036    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d76f9797-13c3-49fa-b018-65de900b8bb3-metrics-tls\") pod \"dns-default-v8q24\" (UID: \"d76f9797-13c3-49fa-b018-65de900b8bb3\") " pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:22.434375 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:22.434337    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:22.757270 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:22.757239    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v8q24"]
Sep 15 12:26:22.760122 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:22.760097    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76f9797_13c3_49fa_b018_65de900b8bb3.slice/crio-f12123b5049a4d7ea9f30de251de87db7b11b8ab5417de87b5609681f1b826e0 WatchSource:0}: Error finding container f12123b5049a4d7ea9f30de251de87db7b11b8ab5417de87b5609681f1b826e0: Status 404 returned error can't find the container with id f12123b5049a4d7ea9f30de251de87db7b11b8ab5417de87b5609681f1b826e0
Sep 15 12:26:22.943083 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:22.943048    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8q24" event={"ID":"d76f9797-13c3-49fa-b018-65de900b8bb3","Type":"ContainerStarted","Data":"f12123b5049a4d7ea9f30de251de87db7b11b8ab5417de87b5609681f1b826e0"}
Sep 15 12:26:24.949998 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:24.949905    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8q24" event={"ID":"d76f9797-13c3-49fa-b018-65de900b8bb3","Type":"ContainerStarted","Data":"cc48e46d83e25939dd416d949165352232020f4c4ca522fd45d6693030c396c4"}
Sep 15 12:26:24.949998 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:24.949962    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8q24" event={"ID":"d76f9797-13c3-49fa-b018-65de900b8bb3","Type":"ContainerStarted","Data":"c9aff55c4f01ffe9de77baec2ae83e60e4ff2061080c968446656b30606c2d0e"}
Sep 15 12:26:24.950491 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:24.950051    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:28.074029 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.073979    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v8q24" podStartSLOduration=20.19579869 podStartE2EDuration="22.073959871s" totalImagesPullingTime="1.878161181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:24.966267367 +0000 UTC m=+52.778178809" watchObservedRunningTime="2026-09-15 12:26:28.073959871 +0000 UTC m=+55.885871313"
Sep 15 12:26:28.074454 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.074193    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"]
Sep 15 12:26:28.075810 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.075795    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"
Sep 15 12:26:28.077897 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.077878    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-s6c8v\""
Sep 15 12:26:28.079020 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.078940    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 15 12:26:28.079020 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.078992    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 15 12:26:28.079172 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.079038    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 15 12:26:28.079172 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.078994    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 15 12:26:28.083111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.081815    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"]
Sep 15 12:26:28.084527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.084512    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.086751 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.086695    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 15 12:26:28.086903 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.086877    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 15 12:26:28.087165 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.087144    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"]
Sep 15 12:26:28.087404 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.087384    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 15 12:26:28.087593 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.087487    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 15 12:26:28.104917 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.104892    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"]
Sep 15 12:26:28.157553 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.157523    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.157680 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.157559    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-ca\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.157680 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.157594    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c116517-22be-40b7-ba32-3f38141313d5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c5bc764-6vlws\" (UID: \"5c116517-22be-40b7-ba32-3f38141313d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"
Sep 15 12:26:28.157680 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.157653    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.157815 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.157680    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2de6591-8a97-45bc-995d-db91f488a975-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.157815 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.157699    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7s54\" (UniqueName: \"kubernetes.io/projected/f2de6591-8a97-45bc-995d-db91f488a975-kube-api-access-f7s54\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.157815 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.157718    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcxxh\" (UniqueName: \"kubernetes.io/projected/5c116517-22be-40b7-ba32-3f38141313d5-kube-api-access-mcxxh\") pod \"managed-serviceaccount-addon-agent-c5bc764-6vlws\" (UID: \"5c116517-22be-40b7-ba32-3f38141313d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"
Sep 15 12:26:28.157815 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.157733    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-hub\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.258152 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.258123    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-ca\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.258336 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.258161    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c116517-22be-40b7-ba32-3f38141313d5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c5bc764-6vlws\" (UID: \"5c116517-22be-40b7-ba32-3f38141313d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"
Sep 15 12:26:28.258336 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.258309    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.258463 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.258340    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2de6591-8a97-45bc-995d-db91f488a975-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.258463 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.258366    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7s54\" (UniqueName: \"kubernetes.io/projected/f2de6591-8a97-45bc-995d-db91f488a975-kube-api-access-f7s54\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.258463 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.258394    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcxxh\" (UniqueName: \"kubernetes.io/projected/5c116517-22be-40b7-ba32-3f38141313d5-kube-api-access-mcxxh\") pod \"managed-serviceaccount-addon-agent-c5bc764-6vlws\" (UID: \"5c116517-22be-40b7-ba32-3f38141313d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"
Sep 15 12:26:28.258463 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.258422    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-hub\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.258659 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.258473    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.259202 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.259178    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2de6591-8a97-45bc-995d-db91f488a975-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.260796 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.260760    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-ca\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.261080 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.261058    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c116517-22be-40b7-ba32-3f38141313d5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c5bc764-6vlws\" (UID: \"5c116517-22be-40b7-ba32-3f38141313d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"
Sep 15 12:26:28.261124 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.261058    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-hub\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.261169 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.261155    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.261216 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.261193    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f2de6591-8a97-45bc-995d-db91f488a975-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.266124 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.266103    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcxxh\" (UniqueName: \"kubernetes.io/projected/5c116517-22be-40b7-ba32-3f38141313d5-kube-api-access-mcxxh\") pod \"managed-serviceaccount-addon-agent-c5bc764-6vlws\" (UID: \"5c116517-22be-40b7-ba32-3f38141313d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"
Sep 15 12:26:28.268369 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.268348    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7s54\" (UniqueName: \"kubernetes.io/projected/f2de6591-8a97-45bc-995d-db91f488a975-kube-api-access-f7s54\") pod \"cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb\" (UID: \"f2de6591-8a97-45bc-995d-db91f488a975\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.401183 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.401115    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"
Sep 15 12:26:28.406343 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.406308    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"
Sep 15 12:26:28.676836 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.676629    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws"]
Sep 15 12:26:28.679429 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:28.679399    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c116517_22be_40b7_ba32_3f38141313d5.slice/crio-0c27cee01d5edcd0dd973a7ed5f820b3a6d5e688523b6b1182d5ff55d1c86b7a WatchSource:0}: Error finding container 0c27cee01d5edcd0dd973a7ed5f820b3a6d5e688523b6b1182d5ff55d1c86b7a: Status 404 returned error can't find the container with id 0c27cee01d5edcd0dd973a7ed5f820b3a6d5e688523b6b1182d5ff55d1c86b7a
Sep 15 12:26:28.679899 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.679880    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb"]
Sep 15 12:26:28.683126 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:28.683104    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2de6591_8a97_45bc_995d_db91f488a975.slice/crio-357a8c84fc7dbdedda26ac2e832cf7717965262e447c4d8f55ff59809068d8da WatchSource:0}: Error finding container 357a8c84fc7dbdedda26ac2e832cf7717965262e447c4d8f55ff59809068d8da: Status 404 returned error can't find the container with id 357a8c84fc7dbdedda26ac2e832cf7717965262e447c4d8f55ff59809068d8da
Sep 15 12:26:28.960765 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.960681    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws" event={"ID":"5c116517-22be-40b7-ba32-3f38141313d5","Type":"ContainerStarted","Data":"0c27cee01d5edcd0dd973a7ed5f820b3a6d5e688523b6b1182d5ff55d1c86b7a"}
Sep 15 12:26:28.961699 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:28.961679    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb" event={"ID":"f2de6591-8a97-45bc-995d-db91f488a975","Type":"ContainerStarted","Data":"357a8c84fc7dbdedda26ac2e832cf7717965262e447c4d8f55ff59809068d8da"}
Sep 15 12:26:31.122844 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.122805    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5jrd8"]
Sep 15 12:26:31.158909 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.158878    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5jrd8"]
Sep 15 12:26:31.159066 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.158976    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.161319 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.161293    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 12:26:31.161431 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.161351    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 12:26:31.161713 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.161694    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 12:26:31.161796 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.161702    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m9kbt\""
Sep 15 12:26:31.162459 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.162430    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 12:26:31.281307 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.281262    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/825edbd7-47cb-4d16-ac7a-34db724a8f13-crio-socket\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.281452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.281326    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/825edbd7-47cb-4d16-ac7a-34db724a8f13-data-volume\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.281452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.281345    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr9jp\" (UniqueName: \"kubernetes.io/projected/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-api-access-nr9jp\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.281452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.281402    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.281452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.281422    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.339403 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.339369    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d5d6d9df4-jjglm"]
Sep 15 12:26:31.353862 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.353831    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.354431 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.354365    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d5d6d9df4-jjglm"]
Sep 15 12:26:31.356414 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.356383    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 15 12:26:31.356540 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.356479    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 15 12:26:31.356540 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.356466    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pkp8g\""
Sep 15 12:26:31.356540 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.356518    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 12:26:31.356674 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.356620    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 12:26:31.356674 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.356634    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 15 12:26:31.356922 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.356904    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 15 12:26:31.357306 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.357284    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 15 12:26:31.382073 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.382016    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.382073 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.382047    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.382219 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.382076    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/825edbd7-47cb-4d16-ac7a-34db724a8f13-crio-socket\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.382219 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:31.382095    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 12:26:31.382219 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.382105    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/825edbd7-47cb-4d16-ac7a-34db724a8f13-data-volume\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.382219 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.382136    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr9jp\" (UniqueName: \"kubernetes.io/projected/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-api-access-nr9jp\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.382219 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:31.382161    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm podName:825edbd7-47cb-4d16-ac7a-34db724a8f13 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:31.882142232 +0000 UTC m=+59.694053675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5jrd8" (UID: "825edbd7-47cb-4d16-ac7a-34db724a8f13") : configmap "kube-rbac-proxy" not found
Sep 15 12:26:31.382219 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:31.382194    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 12:26:31.382430 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:31.382251    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls podName:825edbd7-47cb-4d16-ac7a-34db724a8f13 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:31.882235176 +0000 UTC m=+59.694146609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5jrd8" (UID: "825edbd7-47cb-4d16-ac7a-34db724a8f13") : secret "insights-runtime-extractor-tls" not found
Sep 15 12:26:31.382430 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.382289    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/825edbd7-47cb-4d16-ac7a-34db724a8f13-crio-socket\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.382495 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.382437    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/825edbd7-47cb-4d16-ac7a-34db724a8f13-data-volume\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.393035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.393013    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr9jp\" (UniqueName: \"kubernetes.io/projected/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-api-access-nr9jp\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.483261 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.483230    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-config\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.483436 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.483370    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-oauth-serving-cert\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.483436 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.483410    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-serving-cert\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.483436 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.483434    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-oauth-config\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.483595 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.483457    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-service-ca\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.483595 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.483484    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9qcg\" (UniqueName: \"kubernetes.io/projected/66f19df9-8b91-417d-9655-b6ab91df7f0a-kube-api-access-v9qcg\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.584391 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.584348    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-config\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.584547 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.584451    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-oauth-serving-cert\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.584547 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.584482    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-serving-cert\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.584547 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.584503    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-oauth-config\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.584547 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.584526    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-service-ca\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.584755 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.584549    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9qcg\" (UniqueName: \"kubernetes.io/projected/66f19df9-8b91-417d-9655-b6ab91df7f0a-kube-api-access-v9qcg\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.585530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.585162    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-config\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.585530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.585525    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-service-ca\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.585719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.585580    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-oauth-serving-cert\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.587440 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.587417    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-oauth-config\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.587639 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.587616    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-serving-cert\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.593570 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.593524    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9qcg\" (UniqueName: \"kubernetes.io/projected/66f19df9-8b91-417d-9655-b6ab91df7f0a-kube-api-access-v9qcg\") pod \"console-5d5d6d9df4-jjglm\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") " pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.664706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.664630    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:31.886253 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.886224    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.886253 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:31.886259    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:31.886481 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:31.886401    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 12:26:31.886481 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:31.886453    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls podName:825edbd7-47cb-4d16-ac7a-34db724a8f13 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:32.886439285 +0000 UTC m=+60.698350705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5jrd8" (UID: "825edbd7-47cb-4d16-ac7a-34db724a8f13") : secret "insights-runtime-extractor-tls" not found
Sep 15 12:26:31.886689 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:31.886656    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 12:26:31.886844 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:31.886737    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm podName:825edbd7-47cb-4d16-ac7a-34db724a8f13 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:32.886720161 +0000 UTC m=+60.698631597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5jrd8" (UID: "825edbd7-47cb-4d16-ac7a-34db724a8f13") : configmap "kube-rbac-proxy" not found
Sep 15 12:26:32.512318 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.512296    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v8q24_d76f9797-13c3-49fa-b018-65de900b8bb3/dns/0.log"
Sep 15 12:26:32.691671 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.691649    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v8q24_d76f9797-13c3-49fa-b018-65de900b8bb3/kube-rbac-proxy/0.log"
Sep 15 12:26:32.714624 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.714565    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d5d6d9df4-jjglm"]
Sep 15 12:26:32.717984 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:32.717958    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f19df9_8b91_417d_9655_b6ab91df7f0a.slice/crio-752abf1673d6634433b731231b04979cb59aa6373634d0f8cf6336000e1c3c0b WatchSource:0}: Error finding container 752abf1673d6634433b731231b04979cb59aa6373634d0f8cf6336000e1c3c0b: Status 404 returned error can't find the container with id 752abf1673d6634433b731231b04979cb59aa6373634d0f8cf6336000e1c3c0b
Sep 15 12:26:32.888040 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.888013    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfllx"
Sep 15 12:26:32.895870 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.895846    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:32.896008 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.895882    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:32.896008 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:32.895964    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 12:26:32.896008 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:32.895973    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 12:26:32.896128 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:32.896022    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls podName:825edbd7-47cb-4d16-ac7a-34db724a8f13 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:34.896008943 +0000 UTC m=+62.707920362 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5jrd8" (UID: "825edbd7-47cb-4d16-ac7a-34db724a8f13") : secret "insights-runtime-extractor-tls" not found
Sep 15 12:26:32.896128 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:32.896037    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm podName:825edbd7-47cb-4d16-ac7a-34db724a8f13 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:34.89603059 +0000 UTC m=+62.707942023 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5jrd8" (UID: "825edbd7-47cb-4d16-ac7a-34db724a8f13") : configmap "kube-rbac-proxy" not found
Sep 15 12:26:32.976418 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.976339    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb" event={"ID":"f2de6591-8a97-45bc-995d-db91f488a975","Type":"ContainerStarted","Data":"3b90ed9b1995d96a8bcc0de383eeee35219fca76431311c46d6917999f18040e"}
Sep 15 12:26:32.976418 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.976383    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb" event={"ID":"f2de6591-8a97-45bc-995d-db91f488a975","Type":"ContainerStarted","Data":"9e26c4ad439c1e527ac7dac90904b0b27f491f71c224488a018faf72f38b01f8"}
Sep 15 12:26:32.976418 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.976396    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb" event={"ID":"f2de6591-8a97-45bc-995d-db91f488a975","Type":"ContainerStarted","Data":"37b032ad01c16dfeb1431b276605497876416d9e4bd1823d6dfa6d0139703010"}
Sep 15 12:26:32.977522 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.977499    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5d6d9df4-jjglm" event={"ID":"66f19df9-8b91-417d-9655-b6ab91df7f0a","Type":"ContainerStarted","Data":"752abf1673d6634433b731231b04979cb59aa6373634d0f8cf6336000e1c3c0b"}
Sep 15 12:26:32.998911 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:32.998863    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bff9b9b9f-mkgvb" podStartSLOduration=1.182359258 podStartE2EDuration="4.998850484s" totalImagesPullingTime="3.816491226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:32.99876997 +0000 UTC m=+60.810681411" watchObservedRunningTime="2026-09-15 12:26:32.998850484 +0000 UTC m=+60.810761926"
Sep 15 12:26:33.292377 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.292352    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mqn5j_6b7ec0a1-a68d-4699-bb5d-c3152791a0ef/dns-node-resolver/0.log"
Sep 15 12:26:33.337841 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.337810    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc"]
Sep 15 12:26:33.339956 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.339935    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc"
Sep 15 12:26:33.342018 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.341990    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 15 12:26:33.342018 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.342012    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mgp99\""
Sep 15 12:26:33.347979 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.347951    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc"]
Sep 15 12:26:33.501418 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.501382    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/78bbf197-6b79-4b69-966a-4a81a9c5946b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-2zchc\" (UID: \"78bbf197-6b79-4b69-966a-4a81a9c5946b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc"
Sep 15 12:26:33.602738 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.602657    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/78bbf197-6b79-4b69-966a-4a81a9c5946b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-2zchc\" (UID: \"78bbf197-6b79-4b69-966a-4a81a9c5946b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc"
Sep 15 12:26:33.605568 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.605545    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/78bbf197-6b79-4b69-966a-4a81a9c5946b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-2zchc\" (UID: \"78bbf197-6b79-4b69-966a-4a81a9c5946b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc"
Sep 15 12:26:33.648677 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.648641    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc"
Sep 15 12:26:33.922864 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.922773    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc"]
Sep 15 12:26:33.926430 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:33.926401    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78bbf197_6b79_4b69_966a_4a81a9c5946b.slice/crio-33fd3a95e6d7f5c8f6a402367351474a25c84e73273d2172577bb68e273e805b WatchSource:0}: Error finding container 33fd3a95e6d7f5c8f6a402367351474a25c84e73273d2172577bb68e273e805b: Status 404 returned error can't find the container with id 33fd3a95e6d7f5c8f6a402367351474a25c84e73273d2172577bb68e273e805b
Sep 15 12:26:33.981834 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:33.981602    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc" event={"ID":"78bbf197-6b79-4b69-966a-4a81a9c5946b","Type":"ContainerStarted","Data":"33fd3a95e6d7f5c8f6a402367351474a25c84e73273d2172577bb68e273e805b"}
Sep 15 12:26:34.291348 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:34.291317    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k28bb_2f5cb386-9dde-4e3c-9122-eb1c6042e0e4/node-ca/0.log"
Sep 15 12:26:34.913337 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:34.913299    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:34.913337 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:34.913348    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:34.913956 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:34.913447    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 12:26:34.913956 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:34.913515    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 12:26:34.913956 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:34.913551    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm podName:825edbd7-47cb-4d16-ac7a-34db724a8f13 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:38.913526616 +0000 UTC m=+66.725438056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5jrd8" (UID: "825edbd7-47cb-4d16-ac7a-34db724a8f13") : configmap "kube-rbac-proxy" not found
Sep 15 12:26:34.913956 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:34.913578    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls podName:825edbd7-47cb-4d16-ac7a-34db724a8f13 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:38.913561774 +0000 UTC m=+66.725473208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5jrd8" (UID: "825edbd7-47cb-4d16-ac7a-34db724a8f13") : secret "insights-runtime-extractor-tls" not found
Sep 15 12:26:34.955140 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:34.955114    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v8q24"
Sep 15 12:26:35.292363 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:35.292333    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tp5lb_040b0ab2-3bbc-4355-acf8-b78f7446bdf2/serve-healthcheck-canary/0.log"
Sep 15 12:26:36.992767 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:36.992731    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc" event={"ID":"78bbf197-6b79-4b69-966a-4a81a9c5946b","Type":"ContainerStarted","Data":"2a1ccefcc9849bc3aa66b8be2eb738e505ff8a8ca13dbb243c87e19792785b88"}
Sep 15 12:26:36.993213 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:36.992811    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc"
Sep 15 12:26:36.994512 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:36.994489    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5d6d9df4-jjglm" event={"ID":"66f19df9-8b91-417d-9655-b6ab91df7f0a","Type":"ContainerStarted","Data":"95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3"}
Sep 15 12:26:36.997859 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:36.997841    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc"
Sep 15 12:26:37.007943 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.007896    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-2zchc" podStartSLOduration=1.814715133 podStartE2EDuration="4.007879617s" totalImagesPullingTime="2.193164484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:37.007508774 +0000 UTC m=+64.819420219" watchObservedRunningTime="2026-09-15 12:26:37.007879617 +0000 UTC m=+64.819791050"
Sep 15 12:26:37.041502 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.041443    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d5d6d9df4-jjglm" podStartSLOduration=2.636612455 podStartE2EDuration="6.041423397s" totalImagesPullingTime="3.404810942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:37.041080803 +0000 UTC m=+64.852992244" watchObservedRunningTime="2026-09-15 12:26:37.041423397 +0000 UTC m=+64.853334839"
Sep 15 12:26:37.434795 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.434698    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:26:37.434795 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.434752    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7sq9\" (UniqueName: \"kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9\") pod \"network-check-target-z4d6q\" (UID: \"dbd143a3-56aa-4bef-b0a9-792ba54e447e\") " pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:26:37.434795 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.434769    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:26:37.437143 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.437113    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 12:26:37.437143 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.437133    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 12:26:37.437329 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.437163    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 12:26:37.446945 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.446918    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 12:26:37.447251 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.447228    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca49e661-dd33-4c7c-9751-408033360944-original-pull-secret\") pod \"global-pull-secret-syncer-2n7pd\" (UID: \"ca49e661-dd33-4c7c-9751-408033360944\") " pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:26:37.447329 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.447287    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc48a45a-1993-4cba-a9dc-636f75057239-metrics-certs\") pod \"network-metrics-daemon-rqsvh\" (UID: \"dc48a45a-1993-4cba-a9dc-636f75057239\") " pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:26:37.457949 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.457924    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7sq9\" (UniqueName: \"kubernetes.io/projected/dbd143a3-56aa-4bef-b0a9-792ba54e447e-kube-api-access-r7sq9\") pod \"network-check-target-z4d6q\" (UID: \"dbd143a3-56aa-4bef-b0a9-792ba54e447e\") " pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:26:37.560572 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.560541    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j4plp\""
Sep 15 12:26:37.565502 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.565484    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n7pd"
Sep 15 12:26:37.568598 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.568582    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqsvh"
Sep 15 12:26:37.572801 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.572773    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mnpg9\""
Sep 15 12:26:37.581861 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.581820    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:26:37.866869 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.866839    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2n7pd"]
Sep 15 12:26:37.869676 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:37.869649    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca49e661_dd33_4c7c_9751_408033360944.slice/crio-1a55333f4132f127aa46644c45b9569f61e2b2a2d2c0c4a6a9ce79cf5c7ba74a WatchSource:0}: Error finding container 1a55333f4132f127aa46644c45b9569f61e2b2a2d2c0c4a6a9ce79cf5c7ba74a: Status 404 returned error can't find the container with id 1a55333f4132f127aa46644c45b9569f61e2b2a2d2c0c4a6a9ce79cf5c7ba74a
Sep 15 12:26:37.903622 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.903599    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rqsvh"]
Sep 15 12:26:37.906000 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:37.905965    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc48a45a_1993_4cba_a9dc_636f75057239.slice/crio-22096e76014c0d5776a2c7b32aa89279a298ed6e9170132e0755f014fade705f WatchSource:0}: Error finding container 22096e76014c0d5776a2c7b32aa89279a298ed6e9170132e0755f014fade705f: Status 404 returned error can't find the container with id 22096e76014c0d5776a2c7b32aa89279a298ed6e9170132e0755f014fade705f
Sep 15 12:26:37.934564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.934538    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z4d6q"]
Sep 15 12:26:37.937145 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:37.937094    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbd143a3_56aa_4bef_b0a9_792ba54e447e.slice/crio-62bf36dc830545a7fd459c319e85fdf60cdab7db0126aa41ed0b1187ffce0bea WatchSource:0}: Error finding container 62bf36dc830545a7fd459c319e85fdf60cdab7db0126aa41ed0b1187ffce0bea: Status 404 returned error can't find the container with id 62bf36dc830545a7fd459c319e85fdf60cdab7db0126aa41ed0b1187ffce0bea
Sep 15 12:26:37.998882 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.998850    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z4d6q" event={"ID":"dbd143a3-56aa-4bef-b0a9-792ba54e447e","Type":"ContainerStarted","Data":"62bf36dc830545a7fd459c319e85fdf60cdab7db0126aa41ed0b1187ffce0bea"}
Sep 15 12:26:38.000006 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:37.999984    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqsvh" event={"ID":"dc48a45a-1993-4cba-a9dc-636f75057239","Type":"ContainerStarted","Data":"22096e76014c0d5776a2c7b32aa89279a298ed6e9170132e0755f014fade705f"}
Sep 15 12:26:38.001103 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:38.001078    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2n7pd" event={"ID":"ca49e661-dd33-4c7c-9751-408033360944","Type":"ContainerStarted","Data":"1a55333f4132f127aa46644c45b9569f61e2b2a2d2c0c4a6a9ce79cf5c7ba74a"}
Sep 15 12:26:38.948486 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:38.947815    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:38.948486 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:38.947869    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:38.948486 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:38.948066    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 12:26:38.948486 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:38.948135    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls podName:825edbd7-47cb-4d16-ac7a-34db724a8f13 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:46.948116422 +0000 UTC m=+74.760027846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5jrd8" (UID: "825edbd7-47cb-4d16-ac7a-34db724a8f13") : secret "insights-runtime-extractor-tls" not found
Sep 15 12:26:38.948486 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:38.948155    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 12:26:38.948486 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:38.948283    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm podName:825edbd7-47cb-4d16-ac7a-34db724a8f13 nodeName:}" failed. No retries permitted until 2026-09-15 12:26:46.948263779 +0000 UTC m=+74.760175203 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5jrd8" (UID: "825edbd7-47cb-4d16-ac7a-34db724a8f13") : configmap "kube-rbac-proxy" not found
Sep 15 12:26:40.011176 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:40.011111    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqsvh" event={"ID":"dc48a45a-1993-4cba-a9dc-636f75057239","Type":"ContainerStarted","Data":"009b1da15e6078434d5eb22dd995b564e830aa0a1ce50302fea51eac48c4de8d"}
Sep 15 12:26:40.011176 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:40.011161    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqsvh" event={"ID":"dc48a45a-1993-4cba-a9dc-636f75057239","Type":"ContainerStarted","Data":"bce46fb801ba05dbf3cb241c9fc7f8df23ea1066c784b6ea5898bb4180cf28fb"}
Sep 15 12:26:40.026913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:40.026855    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rqsvh" podStartSLOduration=65.773000844 podStartE2EDuration="1m7.02683065s" totalImagesPullingTime="1.253829806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:40.025118036 +0000 UTC m=+67.837029497" watchObservedRunningTime="2026-09-15 12:26:40.02683065 +0000 UTC m=+67.838742094"
Sep 15 12:26:41.665118 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:41.665078    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:41.665588 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:41.665151    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:41.670968 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:41.670939    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:42.024387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.024354    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:26:42.772832 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.772802    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nptnb"]
Sep 15 12:26:42.774933 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.774917    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"]
Sep 15 12:26:42.775056 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.775039    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.776840 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.776822    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.777313 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777293    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 12:26:42.777418 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777358    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e00a1b6f-a485-437c-938d-70075b2fb20f-metrics-client-ca\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.777418 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777394    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.777504 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777459    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-tls\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.777540 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777498    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-textfile\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.777581 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777550    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-accelerators-collector-config\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.777660 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777643    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wzdhx\""
Sep 15 12:26:42.777717 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777658    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 12:26:42.777717 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777685    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e00a1b6f-a485-437c-938d-70075b2fb20f-root\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.777815 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777738    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jggcd\" (UniqueName: \"kubernetes.io/projected/e00a1b6f-a485-437c-938d-70075b2fb20f-kube-api-access-jggcd\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.777815 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777764    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e00a1b6f-a485-437c-938d-70075b2fb20f-sys\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.777903 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777831    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-wtmp\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.777945 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.777914    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 12:26:42.778239 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.778221    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 12:26:42.778303 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.778248    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 12:26:42.778303 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.778284    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 12:26:42.779020 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.779004    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 15 12:26:42.779090 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.779010    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 15 12:26:42.779144 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.779106    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 15 12:26:42.779144 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.779066    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q2x75\""
Sep 15 12:26:42.788378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.788356    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"]
Sep 15 12:26:42.878094 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878067    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e00a1b6f-a485-437c-938d-70075b2fb20f-metrics-client-ca\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878094 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878100    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/788827a0-a45b-487e-9459-c52473975518-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.878314 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878120    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878314 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878138    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-tls\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878314 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878188    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/788827a0-a45b-487e-9459-c52473975518-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.878314 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878233    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-textfile\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878314 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:42.878237    2586 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 15 12:26:42.878535 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878263    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-accelerators-collector-config\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878535 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:26:42.878339    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-tls podName:e00a1b6f-a485-437c-938d-70075b2fb20f nodeName:}" failed. No retries permitted until 2026-09-15 12:26:43.378312627 +0000 UTC m=+71.190224050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-tls") pod "node-exporter-nptnb" (UID: "e00a1b6f-a485-437c-938d-70075b2fb20f") : secret "node-exporter-tls" not found
Sep 15 12:26:42.878535 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878375    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prr2p\" (UniqueName: \"kubernetes.io/projected/788827a0-a45b-487e-9459-c52473975518-kube-api-access-prr2p\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.878535 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878409    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/788827a0-a45b-487e-9459-c52473975518-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.878535 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878440    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/788827a0-a45b-487e-9459-c52473975518-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878544    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e00a1b6f-a485-437c-938d-70075b2fb20f-root\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878577    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jggcd\" (UniqueName: \"kubernetes.io/projected/e00a1b6f-a485-437c-938d-70075b2fb20f-kube-api-access-jggcd\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878583    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-textfile\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878601    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e00a1b6f-a485-437c-938d-70075b2fb20f-sys\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878642    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e00a1b6f-a485-437c-938d-70075b2fb20f-sys\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878649    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-wtmp\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878591    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e00a1b6f-a485-437c-938d-70075b2fb20f-root\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878684    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/788827a0-a45b-487e-9459-c52473975518-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878732    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e00a1b6f-a485-437c-938d-70075b2fb20f-metrics-client-ca\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878752    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-wtmp\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.878797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.878762    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-accelerators-collector-config\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.880500 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.880483    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.887816 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.887765    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jggcd\" (UniqueName: \"kubernetes.io/projected/e00a1b6f-a485-437c-938d-70075b2fb20f-kube-api-access-jggcd\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:42.979267 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.979231    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/788827a0-a45b-487e-9459-c52473975518-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.979415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.979284    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/788827a0-a45b-487e-9459-c52473975518-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.979415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.979315    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/788827a0-a45b-487e-9459-c52473975518-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.979415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.979342    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prr2p\" (UniqueName: \"kubernetes.io/projected/788827a0-a45b-487e-9459-c52473975518-kube-api-access-prr2p\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.979415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.979360    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/788827a0-a45b-487e-9459-c52473975518-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.979415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.979378    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/788827a0-a45b-487e-9459-c52473975518-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.979841 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.979818    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/788827a0-a45b-487e-9459-c52473975518-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.980154 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.980132    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/788827a0-a45b-487e-9459-c52473975518-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.980194 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.980134    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/788827a0-a45b-487e-9459-c52473975518-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.981841 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.981819    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/788827a0-a45b-487e-9459-c52473975518-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.981841 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.981832    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/788827a0-a45b-487e-9459-c52473975518-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:42.986852 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:42.986831    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prr2p\" (UniqueName: \"kubernetes.io/projected/788827a0-a45b-487e-9459-c52473975518-kube-api-access-prr2p\") pod \"kube-state-metrics-59fbdd77d-qj647\" (UID: \"788827a0-a45b-487e-9459-c52473975518\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:43.023116 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.023040    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z4d6q" event={"ID":"dbd143a3-56aa-4bef-b0a9-792ba54e447e","Type":"ContainerStarted","Data":"e4567225b2fb7705d506844bc990650f70e4c6d70c5f74f195a537076817c8ec"}
Sep 15 12:26:43.023221 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.023189    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:26:43.024600 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.024578    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2n7pd" event={"ID":"ca49e661-dd33-4c7c-9751-408033360944","Type":"ContainerStarted","Data":"36ba2092fa4a734853236710ce947302a2472b65f5edd34fddc84db958b435ba"}
Sep 15 12:26:43.041258 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.041213    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z4d6q" podStartSLOduration=65.605297683 podStartE2EDuration="1m10.041199391s" totalImagesPullingTime="4.435901708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:43.040100082 +0000 UTC m=+70.852011525" watchObservedRunningTime="2026-09-15 12:26:43.041199391 +0000 UTC m=+70.853110867"
Sep 15 12:26:43.053505 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.053461    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2n7pd" podStartSLOduration=65.546500762 podStartE2EDuration="1m10.053444959s" totalImagesPullingTime="4.506944197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:43.05339857 +0000 UTC m=+70.865310016" watchObservedRunningTime="2026-09-15 12:26:43.053444959 +0000 UTC m=+70.865356402"
Sep 15 12:26:43.091750 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.091728    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"
Sep 15 12:26:43.362902 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.362824    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-qj647"]
Sep 15 12:26:43.365991 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:43.365963    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod788827a0_a45b_487e_9459_c52473975518.slice/crio-9020339c077c30884c9b2ae82b88fee700ad2b29a762e097398ac21a2bae7c80 WatchSource:0}: Error finding container 9020339c077c30884c9b2ae82b88fee700ad2b29a762e097398ac21a2bae7c80: Status 404 returned error can't find the container with id 9020339c077c30884c9b2ae82b88fee700ad2b29a762e097398ac21a2bae7c80
Sep 15 12:26:43.382600 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.382570    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-tls\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:43.384689 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.384670    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e00a1b6f-a485-437c-938d-70075b2fb20f-node-exporter-tls\") pod \"node-exporter-nptnb\" (UID: \"e00a1b6f-a485-437c-938d-70075b2fb20f\") " pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:43.384816 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.384804    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nptnb"
Sep 15 12:26:43.395505 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:43.395463    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00a1b6f_a485_437c_938d_70075b2fb20f.slice/crio-a83246e7841ce2f45bb0dbe3d305d96075b9c794d774e09c1e20894bbad171c5 WatchSource:0}: Error finding container a83246e7841ce2f45bb0dbe3d305d96075b9c794d774e09c1e20894bbad171c5: Status 404 returned error can't find the container with id a83246e7841ce2f45bb0dbe3d305d96075b9c794d774e09c1e20894bbad171c5
Sep 15 12:26:43.659509 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:43.659436    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d5d6d9df4-jjglm"]
Sep 15 12:26:44.030315 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:44.030269    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647" event={"ID":"788827a0-a45b-487e-9459-c52473975518","Type":"ContainerStarted","Data":"9020339c077c30884c9b2ae82b88fee700ad2b29a762e097398ac21a2bae7c80"}
Sep 15 12:26:44.031794 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:44.031747    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nptnb" event={"ID":"e00a1b6f-a485-437c-938d-70075b2fb20f","Type":"ContainerStarted","Data":"a83246e7841ce2f45bb0dbe3d305d96075b9c794d774e09c1e20894bbad171c5"}
Sep 15 12:26:45.037238 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:45.037206    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647" event={"ID":"788827a0-a45b-487e-9459-c52473975518","Type":"ContainerStarted","Data":"29249b38ea72061d15824a3d281711a9969bb06df9eb704fd657bc76a36731a9"}
Sep 15 12:26:45.037618 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:45.037268    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647" event={"ID":"788827a0-a45b-487e-9459-c52473975518","Type":"ContainerStarted","Data":"a402e62b307e30a33a53ab93925f5bac9be1f6a4bb56fd0b5f7ee82ca6eed4e8"}
Sep 15 12:26:45.037618 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:45.037289    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647" event={"ID":"788827a0-a45b-487e-9459-c52473975518","Type":"ContainerStarted","Data":"772a395b14435071a6042da47172e8481068b00ef75d250ebd4c8813c1b0d757"}
Sep 15 12:26:45.055385 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:45.055334    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qj647" podStartSLOduration=1.78096175 podStartE2EDuration="3.05531488s" totalImagesPullingTime="1.27435313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:45.054066604 +0000 UTC m=+72.865978048" watchObservedRunningTime="2026-09-15 12:26:45.05531488 +0000 UTC m=+72.867226323"
Sep 15 12:26:47.008589 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.008554    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:47.008589 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.008594    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:47.009245 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.009183    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825edbd7-47cb-4d16-ac7a-34db724a8f13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:47.011143 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.011113    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825edbd7-47cb-4d16-ac7a-34db724a8f13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5jrd8\" (UID: \"825edbd7-47cb-4d16-ac7a-34db724a8f13\") " pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:47.048366 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.048326    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws" event={"ID":"5c116517-22be-40b7-ba32-3f38141313d5","Type":"ContainerStarted","Data":"b35eab7bf89c49cee5f2d2e825737e0170f9f8701598ed265038d39d95f93c7a"}
Sep 15 12:26:47.063273 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.063197    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c5bc764-6vlws" podStartSLOduration=1.274935545 podStartE2EDuration="19.063181278s" totalImagesPullingTime="17.788245733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:47.062580508 +0000 UTC m=+74.874491956" watchObservedRunningTime="2026-09-15 12:26:47.063181278 +0000 UTC m=+74.875092723"
Sep 15 12:26:47.072557 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.072524    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m9kbt\""
Sep 15 12:26:47.081468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.081430    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5jrd8"
Sep 15 12:26:47.281219 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.281141    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-766f4cc575-zr9wp"]
Sep 15 12:26:47.284158 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.284134    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.286632 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.286545    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ceblo6v1j3rqh\""
Sep 15 12:26:47.286909 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.286873    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-z4h9z\""
Sep 15 12:26:47.287031 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.286911    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 15 12:26:47.287031 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.286926    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 15 12:26:47.287031 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.286931    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 15 12:26:47.287279 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.287260    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 15 12:26:47.292543 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.292520    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-766f4cc575-zr9wp"]
Sep 15 12:26:47.310417 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.310387    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-client-ca-bundle\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.310520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.310422    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-metrics-server-audit-profiles\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.310520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.310456    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.310520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.310477    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-secret-metrics-server-client-certs\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.310520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.310494    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfmgr\" (UniqueName: \"kubernetes.io/projected/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-kube-api-access-kfmgr\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.310520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.310512    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-secret-metrics-server-tls\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.310691 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.310556    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-audit-log\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.370388 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.370302    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5jrd8"]
Sep 15 12:26:47.372835 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:47.372813    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825edbd7_47cb_4d16_ac7a_34db724a8f13.slice/crio-501988678e99f901a910200bf1194318fd2d6a9a762ff854a5195982e1bb19e1 WatchSource:0}: Error finding container 501988678e99f901a910200bf1194318fd2d6a9a762ff854a5195982e1bb19e1: Status 404 returned error can't find the container with id 501988678e99f901a910200bf1194318fd2d6a9a762ff854a5195982e1bb19e1
Sep 15 12:26:47.411462 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.411437    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-audit-log\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.411577 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.411481    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-client-ca-bundle\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.411577 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.411519    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-metrics-server-audit-profiles\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.411683 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.411572    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.411683 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.411607    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-secret-metrics-server-client-certs\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.411683 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.411632    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfmgr\" (UniqueName: \"kubernetes.io/projected/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-kube-api-access-kfmgr\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.411683 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.411663    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-secret-metrics-server-tls\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.411883 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.411835    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-audit-log\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.412468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.412446    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.412637 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.412616    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-metrics-server-audit-profiles\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.413910 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.413892    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-secret-metrics-server-tls\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.414541 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.414517    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-secret-metrics-server-client-certs\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.414612 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.414538    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-client-ca-bundle\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.418984 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.418967    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfmgr\" (UniqueName: \"kubernetes.io/projected/9cd3f9dc-3d5a-46cb-966c-ec93090cd379-kube-api-access-kfmgr\") pod \"metrics-server-766f4cc575-zr9wp\" (UID: \"9cd3f9dc-3d5a-46cb-966c-ec93090cd379\") " pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.645861 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.645742    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:26:47.919232 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:47.919160    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-766f4cc575-zr9wp"]
Sep 15 12:26:47.922464 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:47.922440    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cd3f9dc_3d5a_46cb_966c_ec93090cd379.slice/crio-5206c58f375d436ce23ef74988dd4597a7065a8f523542f1e2abf556d722a160 WatchSource:0}: Error finding container 5206c58f375d436ce23ef74988dd4597a7065a8f523542f1e2abf556d722a160: Status 404 returned error can't find the container with id 5206c58f375d436ce23ef74988dd4597a7065a8f523542f1e2abf556d722a160
Sep 15 12:26:48.054060 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:48.054017    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5jrd8" event={"ID":"825edbd7-47cb-4d16-ac7a-34db724a8f13","Type":"ContainerStarted","Data":"fe1ed5d786f409131c5709a7c3a898d17e5c172caa48c93664a25a3d7c400706"}
Sep 15 12:26:48.054499 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:48.054082    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5jrd8" event={"ID":"825edbd7-47cb-4d16-ac7a-34db724a8f13","Type":"ContainerStarted","Data":"501988678e99f901a910200bf1194318fd2d6a9a762ff854a5195982e1bb19e1"}
Sep 15 12:26:48.055701 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:48.055673    2586 generic.go:356] "Generic (PLEG): container finished" podID="e00a1b6f-a485-437c-938d-70075b2fb20f" containerID="39a1a444178201258584ffa9f4fb25a81d53958bad56c96a934494dfe3b2420e" exitCode=0
Sep 15 12:26:48.055899 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:48.055770    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nptnb" event={"ID":"e00a1b6f-a485-437c-938d-70075b2fb20f","Type":"ContainerDied","Data":"39a1a444178201258584ffa9f4fb25a81d53958bad56c96a934494dfe3b2420e"}
Sep 15 12:26:48.057379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:48.057355    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp" event={"ID":"9cd3f9dc-3d5a-46cb-966c-ec93090cd379","Type":"ContainerStarted","Data":"5206c58f375d436ce23ef74988dd4597a7065a8f523542f1e2abf556d722a160"}
Sep 15 12:26:49.063365 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.063327    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5jrd8" event={"ID":"825edbd7-47cb-4d16-ac7a-34db724a8f13","Type":"ContainerStarted","Data":"735fb7b7ac1c85f2375434ab3eece9da734060619e9a2b107ec9995b05370ffd"}
Sep 15 12:26:49.068566 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.068055    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nptnb" event={"ID":"e00a1b6f-a485-437c-938d-70075b2fb20f","Type":"ContainerStarted","Data":"8084a02dfc0cd3249feb2e92b5468c92cb727889f1eafea875dbfe2efb486a53"}
Sep 15 12:26:49.068566 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.068155    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nptnb" event={"ID":"e00a1b6f-a485-437c-938d-70075b2fb20f","Type":"ContainerStarted","Data":"683c469bc153083a5e8a4d63028a38b31ff46b65c2dd655b7ebfec13f5b535b9"}
Sep 15 12:26:49.078383 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.078356    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:26:49.082675 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.082641    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.085712 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.085683    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 15 12:26:49.085712 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.085722    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 15 12:26:49.086020 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.085722    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 15 12:26:49.086137 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086117    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 15 12:26:49.086232 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086166    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 15 12:26:49.086311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086243    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 15 12:26:49.086415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086381    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 15 12:26:49.086415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086406    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9sm22u1afphh3\""
Sep 15 12:26:49.086563 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086418    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 15 12:26:49.086563 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086481    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 15 12:26:49.086563 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086529    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 15 12:26:49.086563 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086535    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fcqmz\""
Sep 15 12:26:49.086757 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086617    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 15 12:26:49.086757 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086638    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 15 12:26:49.086757 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.086645    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 15 12:26:49.089678 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.089657    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 15 12:26:49.093737 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.092100    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nptnb" podStartSLOduration=3.270199647 podStartE2EDuration="7.092083037s" totalImagesPullingTime="3.758516377s" totalInitContainerRuntime="63.367013ms" isStatefulPod=true podCreationTimestamp="2026-09-15 12:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:49.090481687 +0000 UTC m=+76.902393130" watchObservedRunningTime="2026-09-15 12:26:49.092083037 +0000 UTC m=+76.903994481"
Sep 15 12:26:49.101670 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.101646    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:26:49.127054 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127019    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127200 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127064    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127200 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127097    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28ee608a-2903-496a-acf4-de4a9289ab34-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127303 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127228    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127303 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127254    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127303 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127288    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-config\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127467 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127305    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28ee608a-2903-496a-acf4-de4a9289ab34-config-out\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127467 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127327    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127467 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127349    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-web-config\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127467 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127371    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127467 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127423    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127467 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127446    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127748 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127526    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127748 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127573    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bpwc\" (UniqueName: \"kubernetes.io/projected/28ee608a-2903-496a-acf4-de4a9289ab34-kube-api-access-8bpwc\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127748 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127732    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127770    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127820    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127884    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.127928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127917    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.128102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.127942    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229081 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229002    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28ee608a-2903-496a-acf4-de4a9289ab34-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229081 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229056    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229081 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229082    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229115    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-config\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229134    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28ee608a-2903-496a-acf4-de4a9289ab34-config-out\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229154    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229175    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-web-config\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229208    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229239    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229264    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229301    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229369    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bpwc\" (UniqueName: \"kubernetes.io/projected/28ee608a-2903-496a-acf4-de4a9289ab34-kube-api-access-8bpwc\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229398    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229424    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229461    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229509    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229535    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229564    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229620    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.229811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.229653    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.230724 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.230694    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.231743 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.231694    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.233249 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.232923    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.233249 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.232988    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.234793 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.233671    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.234793 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.234023    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.234793 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.234277    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.234793 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.234371    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28ee608a-2903-496a-acf4-de4a9289ab34-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.234793 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.234466    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.234793 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.234678    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28ee608a-2903-496a-acf4-de4a9289ab34-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.236653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.236626    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-web-config\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.237011 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.236947    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.237934 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.237892    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.238040 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.237985    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-config\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.238536 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.238513    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.238630 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.238607    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28ee608a-2903-496a-acf4-de4a9289ab34-config-out\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.239509 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.239466    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.239856 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.239831    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.240289 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.240244    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28ee608a-2903-496a-acf4-de4a9289ab34-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.254556 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.254527    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bpwc\" (UniqueName: \"kubernetes.io/projected/28ee608a-2903-496a-acf4-de4a9289ab34-kube-api-access-8bpwc\") pod \"prometheus-k8s-0\" (UID: \"28ee608a-2903-496a-acf4-de4a9289ab34\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.396872 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.396767    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:26:49.954277 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:49.954199    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:26:49.957831 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:26:49.957798    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ee608a_2903_496a_acf4_de4a9289ab34.slice/crio-f307c218c0f1901af6c4780e830e788241fe3e8de78f0d8cacd8d9fa017aac00 WatchSource:0}: Error finding container f307c218c0f1901af6c4780e830e788241fe3e8de78f0d8cacd8d9fa017aac00: Status 404 returned error can't find the container with id f307c218c0f1901af6c4780e830e788241fe3e8de78f0d8cacd8d9fa017aac00
Sep 15 12:26:50.073956 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:50.073906    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp" event={"ID":"9cd3f9dc-3d5a-46cb-966c-ec93090cd379","Type":"ContainerStarted","Data":"8523b81197ba9f05cefc63c9ba5d15c61b713358d3c91e364f14d371c783ae0b"}
Sep 15 12:26:50.075321 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:50.075291    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28ee608a-2903-496a-acf4-de4a9289ab34","Type":"ContainerStarted","Data":"f307c218c0f1901af6c4780e830e788241fe3e8de78f0d8cacd8d9fa017aac00"}
Sep 15 12:26:50.077499 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:50.077476    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5jrd8" event={"ID":"825edbd7-47cb-4d16-ac7a-34db724a8f13","Type":"ContainerStarted","Data":"d8ecdff4e335ec9f9bbd579a56048f84d4858f88382c8642099ddf753a5ed078"}
Sep 15 12:26:50.090286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:50.090241    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp" podStartSLOduration=1.336484231 podStartE2EDuration="3.090225171s" totalImagesPullingTime="1.75374094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:50.0894987 +0000 UTC m=+77.901410153" watchObservedRunningTime="2026-09-15 12:26:50.090225171 +0000 UTC m=+77.902136679"
Sep 15 12:26:50.105505 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:50.105461    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5jrd8" podStartSLOduration=16.849035512 podStartE2EDuration="19.105445923s" totalImagesPullingTime="2.256410411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:26:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:50.104499474 +0000 UTC m=+77.916410917" watchObservedRunningTime="2026-09-15 12:26:50.105445923 +0000 UTC m=+77.917357364"
Sep 15 12:26:51.084250 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:51.084212    2586 generic.go:356] "Generic (PLEG): container finished" podID="28ee608a-2903-496a-acf4-de4a9289ab34" containerID="2e5054439b5b85357e1c625d336e02f89716a92997c8e5c0c503014e2c75f5d8" exitCode=0
Sep 15 12:26:51.084681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:51.084302    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28ee608a-2903-496a-acf4-de4a9289ab34","Type":"ContainerDied","Data":"2e5054439b5b85357e1c625d336e02f89716a92997c8e5c0c503014e2c75f5d8"}
Sep 15 12:26:54.101059 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:54.100974    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28ee608a-2903-496a-acf4-de4a9289ab34","Type":"ContainerStarted","Data":"68ccb7487fb4228a55c45e3b395fe32afb3658eecef8aedb67321d49f7b027ed"}
Sep 15 12:26:54.101531 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:54.101101    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28ee608a-2903-496a-acf4-de4a9289ab34","Type":"ContainerStarted","Data":"735d626a88ee8476cf4c171dbadb106bc23f0b71c806c5e50e2b9bd11da52ef6"}
Sep 15 12:26:56.112628 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:56.112588    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28ee608a-2903-496a-acf4-de4a9289ab34","Type":"ContainerStarted","Data":"b98a649fb29fb2f5a395b4930bb927919cd0201108de3258a12a052b0b3ddf4f"}
Sep 15 12:26:56.113051 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:56.112655    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28ee608a-2903-496a-acf4-de4a9289ab34","Type":"ContainerStarted","Data":"cd2356ce7a3fd7ded22f63fc9822f08df399b3ba973ef120a41f918735a8fb25"}
Sep 15 12:26:56.113051 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:56.112721    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28ee608a-2903-496a-acf4-de4a9289ab34","Type":"ContainerStarted","Data":"e68eecdd38b2598721335b31f663507855c1210d00058b97dc8702d3f4ebe153"}
Sep 15 12:26:56.113051 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:56.112755    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28ee608a-2903-496a-acf4-de4a9289ab34","Type":"ContainerStarted","Data":"79974b859344f73986afe9a8ee8980bd0af2741700909822486329ff8eb089f2"}
Sep 15 12:26:56.141436 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:56.141374    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.724036394 podStartE2EDuration="7.141351124s" totalImagesPullingTime="5.400679344s" totalInitContainerRuntime="16.635386ms" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:56.139530834 +0000 UTC m=+83.951442316" watchObservedRunningTime="2026-09-15 12:26:56.141351124 +0000 UTC m=+83.953262567"
Sep 15 12:26:59.397141 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:26:59.397097    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:27:07.646769 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:07.646730    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:27:07.647243 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:07.646835    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:27:09.054561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.054497    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d5d6d9df4-jjglm" podUID="66f19df9-8b91-417d-9655-b6ab91df7f0a" containerName="console" containerID="cri-o://95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3" gracePeriod=15
Sep 15 12:27:09.319736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.319677    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:27:09.397247 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.397222    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9qcg\" (UniqueName: \"kubernetes.io/projected/66f19df9-8b91-417d-9655-b6ab91df7f0a-kube-api-access-v9qcg\") pod \"66f19df9-8b91-417d-9655-b6ab91df7f0a\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") "
Sep 15 12:27:09.397400 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.397288    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-oauth-serving-cert\") pod \"66f19df9-8b91-417d-9655-b6ab91df7f0a\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") "
Sep 15 12:27:09.397400 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.397313    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-oauth-config\") pod \"66f19df9-8b91-417d-9655-b6ab91df7f0a\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") "
Sep 15 12:27:09.397480 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.397414    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-config\") pod \"66f19df9-8b91-417d-9655-b6ab91df7f0a\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") "
Sep 15 12:27:09.397522 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.397487    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-serving-cert\") pod \"66f19df9-8b91-417d-9655-b6ab91df7f0a\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") "
Sep 15 12:27:09.397564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.397521    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-service-ca\") pod \"66f19df9-8b91-417d-9655-b6ab91df7f0a\" (UID: \"66f19df9-8b91-417d-9655-b6ab91df7f0a\") "
Sep 15 12:27:09.397774 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.397747    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-config" (OuterVolumeSpecName: "console-config") pod "66f19df9-8b91-417d-9655-b6ab91df7f0a" (UID: "66f19df9-8b91-417d-9655-b6ab91df7f0a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:27:09.397934 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.397744    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "66f19df9-8b91-417d-9655-b6ab91df7f0a" (UID: "66f19df9-8b91-417d-9655-b6ab91df7f0a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:27:09.397997 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.397948    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-service-ca" (OuterVolumeSpecName: "service-ca") pod "66f19df9-8b91-417d-9655-b6ab91df7f0a" (UID: "66f19df9-8b91-417d-9655-b6ab91df7f0a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:27:09.399967 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.399944    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f19df9-8b91-417d-9655-b6ab91df7f0a-kube-api-access-v9qcg" (OuterVolumeSpecName: "kube-api-access-v9qcg") pod "66f19df9-8b91-417d-9655-b6ab91df7f0a" (UID: "66f19df9-8b91-417d-9655-b6ab91df7f0a"). InnerVolumeSpecName "kube-api-access-v9qcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:27:09.400074 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.399963    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "66f19df9-8b91-417d-9655-b6ab91df7f0a" (UID: "66f19df9-8b91-417d-9655-b6ab91df7f0a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:27:09.400074 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.400042    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "66f19df9-8b91-417d-9655-b6ab91df7f0a" (UID: "66f19df9-8b91-417d-9655-b6ab91df7f0a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:27:09.498901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.498864    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-serving-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:27:09.498901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.498896    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-service-ca\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:27:09.498901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.498909    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9qcg\" (UniqueName: \"kubernetes.io/projected/66f19df9-8b91-417d-9655-b6ab91df7f0a-kube-api-access-v9qcg\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:27:09.499128 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.498923    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-oauth-serving-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:27:09.499128 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.498935    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-oauth-config\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:27:09.499128 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:09.498947    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66f19df9-8b91-417d-9655-b6ab91df7f0a-console-config\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:27:10.166010 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:10.165970    2586 generic.go:356] "Generic (PLEG): container finished" podID="66f19df9-8b91-417d-9655-b6ab91df7f0a" containerID="95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3" exitCode=0
Sep 15 12:27:10.166010 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:10.166017    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5d6d9df4-jjglm"
Sep 15 12:27:10.166541 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:10.166056    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5d6d9df4-jjglm" event={"ID":"66f19df9-8b91-417d-9655-b6ab91df7f0a","Type":"ContainerDied","Data":"95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3"}
Sep 15 12:27:10.166541 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:10.166119    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5d6d9df4-jjglm" event={"ID":"66f19df9-8b91-417d-9655-b6ab91df7f0a","Type":"ContainerDied","Data":"752abf1673d6634433b731231b04979cb59aa6373634d0f8cf6336000e1c3c0b"}
Sep 15 12:27:10.166541 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:10.166150    2586 scope.go:117] "RemoveContainer" containerID="95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3"
Sep 15 12:27:10.175142 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:10.175125    2586 scope.go:117] "RemoveContainer" containerID="95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3"
Sep 15 12:27:10.175394 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:27:10.175372    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3\": container with ID starting with 95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3 not found: ID does not exist" containerID="95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3"
Sep 15 12:27:10.175455 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:10.175400    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3"} err="failed to get container status \"95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3\": rpc error: code = NotFound desc = could not find container \"95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3\": container with ID starting with 95dea1b62ecdd194c0098259652926573679aa1eb1c7dbd6a169147c469cc5e3 not found: ID does not exist"
Sep 15 12:27:10.185598 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:10.185576    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d5d6d9df4-jjglm"]
Sep 15 12:27:10.187597 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:10.187577    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d5d6d9df4-jjglm"]
Sep 15 12:27:10.651117 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:10.651087    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66f19df9-8b91-417d-9655-b6ab91df7f0a" path="/var/lib/kubelet/pods/66f19df9-8b91-417d-9655-b6ab91df7f0a/volumes"
Sep 15 12:27:14.034951 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:14.034915    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z4d6q"
Sep 15 12:27:27.651335 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:27.651306    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:27:27.655232 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:27.655209    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-766f4cc575-zr9wp"
Sep 15 12:27:49.397949 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:49.397905    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:27:49.419134 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:49.419108    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:27:50.330541 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:27:50.330507    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:30:32.557017 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:30:32.556985    2586 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 12:31:09.708371 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.708333    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"]
Sep 15 12:31:09.709926 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.708709    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f19df9-8b91-417d-9655-b6ab91df7f0a" containerName="console"
Sep 15 12:31:09.709926 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.708727    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f19df9-8b91-417d-9655-b6ab91df7f0a" containerName="console"
Sep 15 12:31:09.709926 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.708829    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f19df9-8b91-417d-9655-b6ab91df7f0a" containerName="console"
Sep 15 12:31:09.710546 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.710526    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.712964 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.712941    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 15 12:31:09.713060 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.712968    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k9sn2\""
Sep 15 12:31:09.713060 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.712973    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 15 12:31:09.713239 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.713219    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 15 12:31:09.713369 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.713353    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 15 12:31:09.725748 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.725723    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"]
Sep 15 12:31:09.795559 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.795526    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f62ml\" (UniqueName: \"kubernetes.io/projected/2257ca24-da90-4229-a35c-bac604a3d185-kube-api-access-f62ml\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.795749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.795570    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2257ca24-da90-4229-a35c-bac604a3d185-tmp\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.795749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.795593    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2257ca24-da90-4229-a35c-bac604a3d185-manifests\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.795749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.795668    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2257ca24-da90-4229-a35c-bac604a3d185-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.795749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.795742    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2257ca24-da90-4229-a35c-bac604a3d185-webhook-cert\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.896791 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.896740    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f62ml\" (UniqueName: \"kubernetes.io/projected/2257ca24-da90-4229-a35c-bac604a3d185-kube-api-access-f62ml\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.897006 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.896841    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2257ca24-da90-4229-a35c-bac604a3d185-tmp\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.897006 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.896877    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2257ca24-da90-4229-a35c-bac604a3d185-manifests\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.897006 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.896919    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2257ca24-da90-4229-a35c-bac604a3d185-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.897006 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.896980    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2257ca24-da90-4229-a35c-bac604a3d185-webhook-cert\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.897389 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.897363    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2257ca24-da90-4229-a35c-bac604a3d185-manifests\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.899283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.899249    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2257ca24-da90-4229-a35c-bac604a3d185-tmp\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.899463 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.899445    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2257ca24-da90-4229-a35c-bac604a3d185-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.899550 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.899534    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2257ca24-da90-4229-a35c-bac604a3d185-webhook-cert\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:09.910024 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:09.909998    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f62ml\" (UniqueName: \"kubernetes.io/projected/2257ca24-da90-4229-a35c-bac604a3d185-kube-api-access-f62ml\") pod \"opendatahub-operator-controller-manager-665cd58c4f-7dhb5\" (UID: \"2257ca24-da90-4229-a35c-bac604a3d185\") " pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:10.020341 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:10.020310    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:10.296635 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:10.296094    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"]
Sep 15 12:31:10.300834 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:31:10.300808    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2257ca24_da90_4229_a35c_bac604a3d185.slice/crio-29c93bb29d8be097621d860e3fc7713d50f16b206f71102c7b3c840615028fe7 WatchSource:0}: Error finding container 29c93bb29d8be097621d860e3fc7713d50f16b206f71102c7b3c840615028fe7: Status 404 returned error can't find the container with id 29c93bb29d8be097621d860e3fc7713d50f16b206f71102c7b3c840615028fe7
Sep 15 12:31:10.302576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:10.302559    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 12:31:11.019223 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:11.019138    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5" event={"ID":"2257ca24-da90-4229-a35c-bac604a3d185","Type":"ContainerStarted","Data":"29c93bb29d8be097621d860e3fc7713d50f16b206f71102c7b3c840615028fe7"}
Sep 15 12:31:14.033071 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:14.033027    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5" event={"ID":"2257ca24-da90-4229-a35c-bac604a3d185","Type":"ContainerStarted","Data":"7a1e21a7a703fbaa43e53941c8e02dff59b7971aa4614bee03a8bb8f614c5fca"}
Sep 15 12:31:15.038728 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:15.038696    2586 generic.go:356] "Generic (PLEG): container finished" podID="2257ca24-da90-4229-a35c-bac604a3d185" containerID="7a1e21a7a703fbaa43e53941c8e02dff59b7971aa4614bee03a8bb8f614c5fca" exitCode=0
Sep 15 12:31:15.039313 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:15.038750    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5" event={"ID":"2257ca24-da90-4229-a35c-bac604a3d185","Type":"ContainerDied","Data":"7a1e21a7a703fbaa43e53941c8e02dff59b7971aa4614bee03a8bb8f614c5fca"}
Sep 15 12:31:19.055341 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:19.055305    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5" event={"ID":"2257ca24-da90-4229-a35c-bac604a3d185","Type":"ContainerStarted","Data":"95cbf8616919713d9fee0a448ff9f0e81b0b9ebbd5a1b776b23599d95ca9e2df"}
Sep 15 12:31:19.055712 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:19.055391    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:19.077602 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:19.077483    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5" podStartSLOduration=2.445367988 podStartE2EDuration="10.077460103s" totalImagesPullingTime="6.625317396s" totalInitContainerRuntime="1.006774719s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:31:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:31:19.076737744 +0000 UTC m=+346.888649199" watchObservedRunningTime="2026-09-15 12:31:19.077460103 +0000 UTC m=+346.889371546"
Sep 15 12:31:30.061725 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:30.061649    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-665cd58c4f-7dhb5"
Sep 15 12:31:32.354926 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.354889    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"]
Sep 15 12:31:32.360894 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.360877    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.363824 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.363803    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 12:31:32.364681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.364663    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 15 12:31:32.365075 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.365047    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 15 12:31:32.365190 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.365176    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-km5v7\""
Sep 15 12:31:32.365342 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.365319    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 12:31:32.375469 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.375447    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"]
Sep 15 12:31:32.497292 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.497255    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r525\" (UniqueName: \"kubernetes.io/projected/ecdbe354-d2dd-4631-be5c-c08956500a7f-kube-api-access-6r525\") pod \"kube-auth-proxy-6956d8dd68-jj2wn\" (UID: \"ecdbe354-d2dd-4631-be5c-c08956500a7f\") " pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.497292 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.497292    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdbe354-d2dd-4631-be5c-c08956500a7f-tls-certs\") pod \"kube-auth-proxy-6956d8dd68-jj2wn\" (UID: \"ecdbe354-d2dd-4631-be5c-c08956500a7f\") " pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.497523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.497360    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecdbe354-d2dd-4631-be5c-c08956500a7f-tmp\") pod \"kube-auth-proxy-6956d8dd68-jj2wn\" (UID: \"ecdbe354-d2dd-4631-be5c-c08956500a7f\") " pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.598812 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.598760    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdbe354-d2dd-4631-be5c-c08956500a7f-tls-certs\") pod \"kube-auth-proxy-6956d8dd68-jj2wn\" (UID: \"ecdbe354-d2dd-4631-be5c-c08956500a7f\") " pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.598977 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.598842    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecdbe354-d2dd-4631-be5c-c08956500a7f-tmp\") pod \"kube-auth-proxy-6956d8dd68-jj2wn\" (UID: \"ecdbe354-d2dd-4631-be5c-c08956500a7f\") " pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.598977 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.598940    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r525\" (UniqueName: \"kubernetes.io/projected/ecdbe354-d2dd-4631-be5c-c08956500a7f-kube-api-access-6r525\") pod \"kube-auth-proxy-6956d8dd68-jj2wn\" (UID: \"ecdbe354-d2dd-4631-be5c-c08956500a7f\") " pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.601147 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.601124    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 15 12:31:32.601317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.601297    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecdbe354-d2dd-4631-be5c-c08956500a7f-tmp\") pod \"kube-auth-proxy-6956d8dd68-jj2wn\" (UID: \"ecdbe354-d2dd-4631-be5c-c08956500a7f\") " pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.606126 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.606057    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 12:31:32.611830 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.611808    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdbe354-d2dd-4631-be5c-c08956500a7f-tls-certs\") pod \"kube-auth-proxy-6956d8dd68-jj2wn\" (UID: \"ecdbe354-d2dd-4631-be5c-c08956500a7f\") " pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.616649 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.616623    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 12:31:32.627268 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.627238    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r525\" (UniqueName: \"kubernetes.io/projected/ecdbe354-d2dd-4631-be5c-c08956500a7f-kube-api-access-6r525\") pod \"kube-auth-proxy-6956d8dd68-jj2wn\" (UID: \"ecdbe354-d2dd-4631-be5c-c08956500a7f\") " pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.672393 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.672365    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-km5v7\""
Sep 15 12:31:32.675942 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.675920    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c5f54574c-v9kl9"]
Sep 15 12:31:32.679270 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.679248    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.680274 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.680258    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"
Sep 15 12:31:32.682156 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.681845    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 15 12:31:32.682156 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.681908    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 12:31:32.682267 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.682163    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 15 12:31:32.682267 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.682163    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pkp8g\""
Sep 15 12:31:32.682365 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.682265    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 15 12:31:32.682418 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.682380    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 12:31:32.683472 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.683451    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 15 12:31:32.684810 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.684768    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 15 12:31:32.687969 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.687949    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 15 12:31:32.694916 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.694853    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c5f54574c-v9kl9"]
Sep 15 12:31:32.800327 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.800295    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-console-serving-cert\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.800327 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.800329    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-console-oauth-config\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.800587 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.800349    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwcrn\" (UniqueName: \"kubernetes.io/projected/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-kube-api-access-nwcrn\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.800587 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.800557    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-oauth-serving-cert\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.800705 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.800621    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-service-ca\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.800705 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.800646    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-trusted-ca-bundle\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.800892 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.800852    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-console-config\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.902025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.901942    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-oauth-serving-cert\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.902025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.901984    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-service-ca\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.902025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.902003    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-trusted-ca-bundle\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.902287 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.902039    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-console-config\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.902287 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.902055    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-console-serving-cert\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.902287 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.902070    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-console-oauth-config\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.902287 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.902085    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwcrn\" (UniqueName: \"kubernetes.io/projected/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-kube-api-access-nwcrn\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.902868 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.902840    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-oauth-serving-cert\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.902985 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.902837    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-console-config\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.903040 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.902994    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-service-ca\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.903040 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.903033    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-trusted-ca-bundle\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.904523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.904505    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-console-serving-cert\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.905528 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.905508    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-console-oauth-config\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.910342 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.910321    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwcrn\" (UniqueName: \"kubernetes.io/projected/bb7a6636-e72e-4a8e-aadc-f3d253df10e4-kube-api-access-nwcrn\") pod \"console-c5f54574c-v9kl9\" (UID: \"bb7a6636-e72e-4a8e-aadc-f3d253df10e4\") " pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:32.958845 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.958812    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn"]
Sep 15 12:31:32.961623 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:31:32.961592    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecdbe354_d2dd_4631_be5c_c08956500a7f.slice/crio-9e88d1ec76c05b4c65acde96994b1b4bf200a4993ebcd19ea2e9f041081e2eb3 WatchSource:0}: Error finding container 9e88d1ec76c05b4c65acde96994b1b4bf200a4993ebcd19ea2e9f041081e2eb3: Status 404 returned error can't find the container with id 9e88d1ec76c05b4c65acde96994b1b4bf200a4993ebcd19ea2e9f041081e2eb3
Sep 15 12:31:32.989007 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:32.988967    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:33.108337 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.108303    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn" event={"ID":"ecdbe354-d2dd-4631-be5c-c08956500a7f","Type":"ContainerStarted","Data":"9e88d1ec76c05b4c65acde96994b1b4bf200a4993ebcd19ea2e9f041081e2eb3"}
Sep 15 12:31:33.274742 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.274710    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c5f54574c-v9kl9"]
Sep 15 12:31:33.277509 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:31:33.277476    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb7a6636_e72e_4a8e_aadc_f3d253df10e4.slice/crio-0e680c607a4491cc7d6238b1d5fc06b0bfbafa74093b2872b857932b0b635cd6 WatchSource:0}: Error finding container 0e680c607a4491cc7d6238b1d5fc06b0bfbafa74093b2872b857932b0b635cd6: Status 404 returned error can't find the container with id 0e680c607a4491cc7d6238b1d5fc06b0bfbafa74093b2872b857932b0b635cd6
Sep 15 12:31:33.589014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.588918    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"]
Sep 15 12:31:33.591677 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.591652    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.594086 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.593980    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 15 12:31:33.594086 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.594015    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 15 12:31:33.594086 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.594042    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 15 12:31:33.594086 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.594015    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 15 12:31:33.594355 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.594152    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fwc8b\""
Sep 15 12:31:33.607114 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.607090    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"]
Sep 15 12:31:33.708955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.708911    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.709149 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.708973    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf2fh\" (UniqueName: \"kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-kube-api-access-vf2fh\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.709149 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.709065    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.709149 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.709109    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.709286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.709196    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.709286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.709268    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.709368 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.709307    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.810409 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.810234    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.810409 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.810294    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf2fh\" (UniqueName: \"kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-kube-api-access-vf2fh\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.810409 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.810355    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.810409 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.810392    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.810752 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.810438    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.810752 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.810484    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.810752 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.810513    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.811653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.811623    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.813810 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.813491    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.813929 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.813854    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.814556 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.814527    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.814730 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.814707    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.819284 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.819262    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.820271 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.820249    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf2fh\" (UniqueName: \"kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-kube-api-access-vf2fh\") pod \"istiod-openshift-gateway-7b7995d5cd-j5xvd\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:33.903361 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:33.903276    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:34.115408 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:34.115361    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c5f54574c-v9kl9" event={"ID":"bb7a6636-e72e-4a8e-aadc-f3d253df10e4","Type":"ContainerStarted","Data":"6103ebc8fc8e530acbc2e77d284660cea925738560b99c3b74a122d0c5ad1cd5"}
Sep 15 12:31:34.115597 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:34.115431    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c5f54574c-v9kl9" event={"ID":"bb7a6636-e72e-4a8e-aadc-f3d253df10e4","Type":"ContainerStarted","Data":"0e680c607a4491cc7d6238b1d5fc06b0bfbafa74093b2872b857932b0b635cd6"}
Sep 15 12:31:34.145011 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:34.144936    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c5f54574c-v9kl9" podStartSLOduration=2.144879726 podStartE2EDuration="2.144879726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:31:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:31:34.142295033 +0000 UTC m=+361.954206500" watchObservedRunningTime="2026-09-15 12:31:34.144879726 +0000 UTC m=+361.956791170"
Sep 15 12:31:34.195002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:34.194935    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"]
Sep 15 12:31:34.197643 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:31:34.197605    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e80d223_ffe3_4fb5_bd24_a6b6a8f5780c.slice/crio-bdfd7205cf826ce6a006878860a865db35c00b60df61f0093818d71cd8f7d5fe WatchSource:0}: Error finding container bdfd7205cf826ce6a006878860a865db35c00b60df61f0093818d71cd8f7d5fe: Status 404 returned error can't find the container with id bdfd7205cf826ce6a006878860a865db35c00b60df61f0093818d71cd8f7d5fe
Sep 15 12:31:35.121428 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:35.121393    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd" event={"ID":"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c","Type":"ContainerStarted","Data":"bdfd7205cf826ce6a006878860a865db35c00b60df61f0093818d71cd8f7d5fe"}
Sep 15 12:31:35.694062 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:35.694029    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 15 12:31:36.127207 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:36.127172    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn" event={"ID":"ecdbe354-d2dd-4631-be5c-c08956500a7f","Type":"ContainerStarted","Data":"e35c2095ea638dfcef73b83e0182ca22531f06efc538ff38bfbbbbce99d29834"}
Sep 15 12:31:36.143427 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:36.143366    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6956d8dd68-jj2wn" podStartSLOduration=1.4152060739999999 podStartE2EDuration="4.143346183s" totalImagesPullingTime="2.728140109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:31:36.142850536 +0000 UTC m=+363.954761980" watchObservedRunningTime="2026-09-15 12:31:36.143346183 +0000 UTC m=+363.955257626"
Sep 15 12:31:39.894749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:39.894708    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:31:39.894998 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:39.894801    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:31:40.148426 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:40.148334    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd" event={"ID":"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c","Type":"ContainerStarted","Data":"759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd"}
Sep 15 12:31:40.148583 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:40.148517    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:40.150180 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:40.150146    2586 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-j5xvd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 12:31:40.150319 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:40.150197    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd" podUID="6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 12:31:40.166118 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:40.166075    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd" podStartSLOduration=1.471886769 podStartE2EDuration="7.166060711s" totalImagesPullingTime="5.694173942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:31:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:31:40.164726885 +0000 UTC m=+367.976638328" watchObservedRunningTime="2026-09-15 12:31:40.166060711 +0000 UTC m=+367.977972152"
Sep 15 12:31:41.152730 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:41.152700    2586 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-j5xvd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 12:31:41.153129 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:41.152749    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd" podUID="6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 12:31:42.989939 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:42.989896    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:42.990364 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:42.989973    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:42.995046 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:42.995020    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:43.163144 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:43.163112    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c5f54574c-v9kl9"
Sep 15 12:31:44.153914 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:44.153886    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:51.887935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.887897    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-784kg"]
Sep 15 12:31:51.891312 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.891287    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg"
Sep 15 12:31:51.893824 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.893799    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-smlqg\""
Sep 15 12:31:51.893966 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.893804    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 15 12:31:51.897200 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.897176    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"]
Sep 15 12:31:51.900975 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.900957    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"
Sep 15 12:31:51.903397 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.903375    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 15 12:31:51.903397 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.903390    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6l45m\""
Sep 15 12:31:51.903703 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.903685    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-784kg"]
Sep 15 12:31:51.913378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.913351    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"]
Sep 15 12:31:51.969294 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.969228    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c095478-39f7-402e-8be0-37aac021dda2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vg9wm\" (UID: \"1c095478-39f7-402e-8be0-37aac021dda2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"
Sep 15 12:31:51.969457 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.969335    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf6tr\" (UniqueName: \"kubernetes.io/projected/1c095478-39f7-402e-8be0-37aac021dda2-kube-api-access-bf6tr\") pod \"llmisvc-controller-manager-777c6bd7d5-vg9wm\" (UID: \"1c095478-39f7-402e-8be0-37aac021dda2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"
Sep 15 12:31:51.969457 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.969372    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52dm8\" (UniqueName: \"kubernetes.io/projected/c060ad4e-12da-469f-8661-ef92971fe8ad-kube-api-access-52dm8\") pod \"kserve-controller-manager-d6f4b65b8-784kg\" (UID: \"c060ad4e-12da-469f-8661-ef92971fe8ad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg"
Sep 15 12:31:51.969588 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:51.969461    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c060ad4e-12da-469f-8661-ef92971fe8ad-cert\") pod \"kserve-controller-manager-d6f4b65b8-784kg\" (UID: \"c060ad4e-12da-469f-8661-ef92971fe8ad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg"
Sep 15 12:31:52.070225 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.070182    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c095478-39f7-402e-8be0-37aac021dda2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vg9wm\" (UID: \"1c095478-39f7-402e-8be0-37aac021dda2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"
Sep 15 12:31:52.070225 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.070231    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf6tr\" (UniqueName: \"kubernetes.io/projected/1c095478-39f7-402e-8be0-37aac021dda2-kube-api-access-bf6tr\") pod \"llmisvc-controller-manager-777c6bd7d5-vg9wm\" (UID: \"1c095478-39f7-402e-8be0-37aac021dda2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"
Sep 15 12:31:52.070447 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.070256    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dm8\" (UniqueName: \"kubernetes.io/projected/c060ad4e-12da-469f-8661-ef92971fe8ad-kube-api-access-52dm8\") pod \"kserve-controller-manager-d6f4b65b8-784kg\" (UID: \"c060ad4e-12da-469f-8661-ef92971fe8ad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg"
Sep 15 12:31:52.070447 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.070318    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c060ad4e-12da-469f-8661-ef92971fe8ad-cert\") pod \"kserve-controller-manager-d6f4b65b8-784kg\" (UID: \"c060ad4e-12da-469f-8661-ef92971fe8ad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg"
Sep 15 12:31:52.072816 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.072773    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c095478-39f7-402e-8be0-37aac021dda2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vg9wm\" (UID: \"1c095478-39f7-402e-8be0-37aac021dda2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"
Sep 15 12:31:52.072969 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.072926    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c060ad4e-12da-469f-8661-ef92971fe8ad-cert\") pod \"kserve-controller-manager-d6f4b65b8-784kg\" (UID: \"c060ad4e-12da-469f-8661-ef92971fe8ad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg"
Sep 15 12:31:52.087377 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.087350    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf6tr\" (UniqueName: \"kubernetes.io/projected/1c095478-39f7-402e-8be0-37aac021dda2-kube-api-access-bf6tr\") pod \"llmisvc-controller-manager-777c6bd7d5-vg9wm\" (UID: \"1c095478-39f7-402e-8be0-37aac021dda2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"
Sep 15 12:31:52.089452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.089426    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52dm8\" (UniqueName: \"kubernetes.io/projected/c060ad4e-12da-469f-8661-ef92971fe8ad-kube-api-access-52dm8\") pod \"kserve-controller-manager-d6f4b65b8-784kg\" (UID: \"c060ad4e-12da-469f-8661-ef92971fe8ad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg"
Sep 15 12:31:52.203038 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.202951    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg"
Sep 15 12:31:52.213269 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.213239    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"
Sep 15 12:31:52.476542 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.476432    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-784kg"]
Sep 15 12:31:52.479323 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:31:52.479287    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc060ad4e_12da_469f_8661_ef92971fe8ad.slice/crio-5bab660f71caee67c2698d6a44ce896d76e57da31c549e6eb89f88e02f4c925f WatchSource:0}: Error finding container 5bab660f71caee67c2698d6a44ce896d76e57da31c549e6eb89f88e02f4c925f: Status 404 returned error can't find the container with id 5bab660f71caee67c2698d6a44ce896d76e57da31c549e6eb89f88e02f4c925f
Sep 15 12:31:52.491048 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:52.491023    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"]
Sep 15 12:31:52.493751 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:31:52.493726    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1c095478_39f7_402e_8be0_37aac021dda2.slice/crio-e3445e4477826e4dc27a838f227805d1dbc9593065bcc1b04962f8631fcc75ae WatchSource:0}: Error finding container e3445e4477826e4dc27a838f227805d1dbc9593065bcc1b04962f8631fcc75ae: Status 404 returned error can't find the container with id e3445e4477826e4dc27a838f227805d1dbc9593065bcc1b04962f8631fcc75ae
Sep 15 12:31:53.179627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.177815    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"]
Sep 15 12:31:53.179627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.178049    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd" podUID="6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" containerName="discovery" containerID="cri-o://759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd" gracePeriod=30
Sep 15 12:31:53.200688 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.200646    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg" event={"ID":"c060ad4e-12da-469f-8661-ef92971fe8ad","Type":"ContainerStarted","Data":"5bab660f71caee67c2698d6a44ce896d76e57da31c549e6eb89f88e02f4c925f"}
Sep 15 12:31:53.202846 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.202811    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm" event={"ID":"1c095478-39f7-402e-8be0-37aac021dda2","Type":"ContainerStarted","Data":"e3445e4477826e4dc27a838f227805d1dbc9593065bcc1b04962f8631fcc75ae"}
Sep 15 12:31:53.545651 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.545621    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:53.590526 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.590489    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-dns-cert\") pod \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") "
Sep 15 12:31:53.590526 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.590534    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-local-certs\") pod \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") "
Sep 15 12:31:53.590818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.590586    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-kubeconfig\") pod \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") "
Sep 15 12:31:53.590818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.590641    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf2fh\" (UniqueName: \"kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-kube-api-access-vf2fh\") pod \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") "
Sep 15 12:31:53.590818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.590664    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-token\") pod \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") "
Sep 15 12:31:53.590818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.590690    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-ca-configmap\") pod \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") "
Sep 15 12:31:53.590818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.590721    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-cacerts\") pod \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\" (UID: \"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c\") "
Sep 15 12:31:53.591685 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.591638    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" (UID: "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:31:53.593907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.593876    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" (UID: "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:31:53.600029 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.599927    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-cacerts" (OuterVolumeSpecName: "cacerts") pod "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" (UID: "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:31:53.600029 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.599958    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" (UID: "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:31:53.600029 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.599979    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-kube-api-access-vf2fh" (OuterVolumeSpecName: "kube-api-access-vf2fh") pod "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" (UID: "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c"). InnerVolumeSpecName "kube-api-access-vf2fh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:31:53.600029 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.600004    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" (UID: "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:31:53.602247 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.602207    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-local-certs" (OuterVolumeSpecName: "local-certs") pod "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" (UID: "6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:31:53.691736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.691698    2586 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-kubeconfig\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:31:53.691736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.691738    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf2fh\" (UniqueName: \"kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-kube-api-access-vf2fh\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:31:53.691986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.691756    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:31:53.691986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.691770    2586 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-ca-configmap\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:31:53.691986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.691797    2586 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-cacerts\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:31:53.691986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.691811    2586 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-istio-csr-dns-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:31:53.691986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:53.691824    2586 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c-local-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:31:54.209987 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:54.209950    2586 generic.go:356] "Generic (PLEG): container finished" podID="6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" containerID="759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd" exitCode=0
Sep 15 12:31:54.210422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:54.210009    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"
Sep 15 12:31:54.210422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:54.210020    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd" event={"ID":"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c","Type":"ContainerDied","Data":"759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd"}
Sep 15 12:31:54.210422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:54.210075    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd" event={"ID":"6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c","Type":"ContainerDied","Data":"bdfd7205cf826ce6a006878860a865db35c00b60df61f0093818d71cd8f7d5fe"}
Sep 15 12:31:54.210422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:54.210109    2586 scope.go:117] "RemoveContainer" containerID="759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd"
Sep 15 12:31:54.235024 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:54.234969    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"]
Sep 15 12:31:54.241713 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:54.241679    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j5xvd"]
Sep 15 12:31:54.653890 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:54.653850    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" path="/var/lib/kubelet/pods/6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c/volumes"
Sep 15 12:31:55.376948 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:55.376760    2586 scope.go:117] "RemoveContainer" containerID="759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd"
Sep 15 12:31:55.377282 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:31:55.377256    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd\": container with ID starting with 759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd not found: ID does not exist" containerID="759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd"
Sep 15 12:31:55.377353 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:55.377293    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd"} err="failed to get container status \"759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd\": rpc error: code = NotFound desc = could not find container \"759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd\": container with ID starting with 759adb919a6ed6e3818d4cc0bce68fe3f7739eb162e4ccd6133b40afed9643dd not found: ID does not exist"
Sep 15 12:31:56.222490 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:56.222450    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg" event={"ID":"c060ad4e-12da-469f-8661-ef92971fe8ad","Type":"ContainerStarted","Data":"dc95926d160e69ac6e1360547a423ac6e29e37b29086b7dfe9eaab8f222b521f"}
Sep 15 12:31:56.222701 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:56.222652    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg"
Sep 15 12:31:56.224041 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:56.224012    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm" event={"ID":"1c095478-39f7-402e-8be0-37aac021dda2","Type":"ContainerStarted","Data":"ab9895f59d9dfe694fad2284ee85880fc170c527b197b76ba99654d0b3b5abc4"}
Sep 15 12:31:56.224167 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:56.224149    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"
Sep 15 12:31:56.240771 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:56.240727    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg" podStartSLOduration=2.33513831 podStartE2EDuration="5.240712056s" totalImagesPullingTime="2.905573746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:31:56.239010255 +0000 UTC m=+384.050921697" watchObservedRunningTime="2026-09-15 12:31:56.240712056 +0000 UTC m=+384.052623497"
Sep 15 12:31:56.255824 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:31:56.255758    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm" podStartSLOduration=2.359850238 podStartE2EDuration="5.255743888s" totalImagesPullingTime="2.89589365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:31:56.254772744 +0000 UTC m=+384.066684187" watchObservedRunningTime="2026-09-15 12:31:56.255743888 +0000 UTC m=+384.067655330"
Sep 15 12:32:27.236062 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:32:27.236034    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vg9wm"
Sep 15 12:32:27.239051 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:32:27.239029    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-784kg"
Sep 15 12:33:12.714738 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.714699    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-s7dhm"]
Sep 15 12:33:12.715249 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.715046    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" containerName="discovery"
Sep 15 12:33:12.715249 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.715059    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" containerName="discovery"
Sep 15 12:33:12.715249 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.715130    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e80d223-ffe3-4fb5-bd24-a6b6a8f5780c" containerName="discovery"
Sep 15 12:33:12.717339 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.717317    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-s7dhm"
Sep 15 12:33:12.720156 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.720129    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 15 12:33:12.720286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.720162    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 15 12:33:12.720286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.720206    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5f768\""
Sep 15 12:33:12.732390 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.732365    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-s7dhm"]
Sep 15 12:33:12.881361 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.881323    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zscd2\" (UniqueName: \"kubernetes.io/projected/0840121f-8970-4437-82e1-c0eb8ec392b8-kube-api-access-zscd2\") pod \"authorino-operator-76b478bd78-s7dhm\" (UID: \"0840121f-8970-4437-82e1-c0eb8ec392b8\") " pod="kuadrant-system/authorino-operator-76b478bd78-s7dhm"
Sep 15 12:33:12.982511 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.982483    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zscd2\" (UniqueName: \"kubernetes.io/projected/0840121f-8970-4437-82e1-c0eb8ec392b8-kube-api-access-zscd2\") pod \"authorino-operator-76b478bd78-s7dhm\" (UID: \"0840121f-8970-4437-82e1-c0eb8ec392b8\") " pod="kuadrant-system/authorino-operator-76b478bd78-s7dhm"
Sep 15 12:33:12.993772 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:12.993735    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zscd2\" (UniqueName: \"kubernetes.io/projected/0840121f-8970-4437-82e1-c0eb8ec392b8-kube-api-access-zscd2\") pod \"authorino-operator-76b478bd78-s7dhm\" (UID: \"0840121f-8970-4437-82e1-c0eb8ec392b8\") " pod="kuadrant-system/authorino-operator-76b478bd78-s7dhm"
Sep 15 12:33:13.029223 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:13.029197    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-s7dhm"
Sep 15 12:33:13.300935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:13.300911    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-s7dhm"]
Sep 15 12:33:13.303514 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:33:13.303479    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0840121f_8970_4437_82e1_c0eb8ec392b8.slice/crio-2ad5b6f7eebfdae7eaf0a669f72841251eb5cc04c9b4bb621c30efa119b17e9e WatchSource:0}: Error finding container 2ad5b6f7eebfdae7eaf0a669f72841251eb5cc04c9b4bb621c30efa119b17e9e: Status 404 returned error can't find the container with id 2ad5b6f7eebfdae7eaf0a669f72841251eb5cc04c9b4bb621c30efa119b17e9e
Sep 15 12:33:13.540237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:13.540205    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-s7dhm" event={"ID":"0840121f-8970-4437-82e1-c0eb8ec392b8","Type":"ContainerStarted","Data":"2ad5b6f7eebfdae7eaf0a669f72841251eb5cc04c9b4bb621c30efa119b17e9e"}
Sep 15 12:33:14.913863 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:14.913831    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m"]
Sep 15 12:33:14.916745 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:14.916728    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m"
Sep 15 12:33:14.918956 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:14.918933    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 15 12:33:14.919066 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:14.918957    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hfqpt\""
Sep 15 12:33:14.928213 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:14.928185    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m"]
Sep 15 12:33:15.000076 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:15.000037    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgq5w\" (UniqueName: \"kubernetes.io/projected/ad50bc5f-fac3-4fd3-ac1d-4a8455b42f3c-kube-api-access-dgq5w\") pod \"dns-operator-controller-manager-7b84cd7c9c-snq5m\" (UID: \"ad50bc5f-fac3-4fd3-ac1d-4a8455b42f3c\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m"
Sep 15 12:33:15.101541 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:15.101498    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgq5w\" (UniqueName: \"kubernetes.io/projected/ad50bc5f-fac3-4fd3-ac1d-4a8455b42f3c-kube-api-access-dgq5w\") pod \"dns-operator-controller-manager-7b84cd7c9c-snq5m\" (UID: \"ad50bc5f-fac3-4fd3-ac1d-4a8455b42f3c\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m"
Sep 15 12:33:15.119172 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:15.119143    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgq5w\" (UniqueName: \"kubernetes.io/projected/ad50bc5f-fac3-4fd3-ac1d-4a8455b42f3c-kube-api-access-dgq5w\") pod \"dns-operator-controller-manager-7b84cd7c9c-snq5m\" (UID: \"ad50bc5f-fac3-4fd3-ac1d-4a8455b42f3c\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m"
Sep 15 12:33:15.233364 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:15.233334    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m"
Sep 15 12:33:15.507436 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:15.507310    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m"]
Sep 15 12:33:15.511376 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:33:15.511347    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad50bc5f_fac3_4fd3_ac1d_4a8455b42f3c.slice/crio-74a99db8a3f7c88265583f44be8fd9f13d3acf2ccbf3eb6154a06c7b3b5a945f WatchSource:0}: Error finding container 74a99db8a3f7c88265583f44be8fd9f13d3acf2ccbf3eb6154a06c7b3b5a945f: Status 404 returned error can't find the container with id 74a99db8a3f7c88265583f44be8fd9f13d3acf2ccbf3eb6154a06c7b3b5a945f
Sep 15 12:33:15.549795 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:15.549747    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m" event={"ID":"ad50bc5f-fac3-4fd3-ac1d-4a8455b42f3c","Type":"ContainerStarted","Data":"74a99db8a3f7c88265583f44be8fd9f13d3acf2ccbf3eb6154a06c7b3b5a945f"}
Sep 15 12:33:19.374838 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.374803    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"]
Sep 15 12:33:19.378366 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.378346    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:19.380881 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.380863    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f4mdv\""
Sep 15 12:33:19.393187 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.393163    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"]
Sep 15 12:33:19.544044 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.544008    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d87d9cce-2585-4025-b0f6-71bf6228012b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" (UID: \"d87d9cce-2585-4025-b0f6-71bf6228012b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:19.544220 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.544059    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxqw7\" (UniqueName: \"kubernetes.io/projected/d87d9cce-2585-4025-b0f6-71bf6228012b-kube-api-access-wxqw7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" (UID: \"d87d9cce-2585-4025-b0f6-71bf6228012b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:19.572773 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.572741    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-s7dhm" event={"ID":"0840121f-8970-4437-82e1-c0eb8ec392b8","Type":"ContainerStarted","Data":"5e4e619964fa7b50ab7e2d0c91be62d1f6a4a03c179cfd6866846204b5dd8067"}
Sep 15 12:33:19.572947 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.572866    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-s7dhm"
Sep 15 12:33:19.574485 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.574461    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m" event={"ID":"ad50bc5f-fac3-4fd3-ac1d-4a8455b42f3c","Type":"ContainerStarted","Data":"54e50946415b20e7af533349a3a664c829b20aed4d6267b72f6139676e4851af"}
Sep 15 12:33:19.574600 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.574589    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m"
Sep 15 12:33:19.589580 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.589534    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-s7dhm" podStartSLOduration=2.369824115 podStartE2EDuration="7.589520628s" totalImagesPullingTime="5.219696513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:33:19.588486444 +0000 UTC m=+467.400397886" watchObservedRunningTime="2026-09-15 12:33:19.589520628 +0000 UTC m=+467.401432070"
Sep 15 12:33:19.644802 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.644685    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d87d9cce-2585-4025-b0f6-71bf6228012b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" (UID: \"d87d9cce-2585-4025-b0f6-71bf6228012b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:19.644802 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.644729    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxqw7\" (UniqueName: \"kubernetes.io/projected/d87d9cce-2585-4025-b0f6-71bf6228012b-kube-api-access-wxqw7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" (UID: \"d87d9cce-2585-4025-b0f6-71bf6228012b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:19.645203 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.645178    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d87d9cce-2585-4025-b0f6-71bf6228012b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" (UID: \"d87d9cce-2585-4025-b0f6-71bf6228012b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:19.655045 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.655012    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxqw7\" (UniqueName: \"kubernetes.io/projected/d87d9cce-2585-4025-b0f6-71bf6228012b-kube-api-access-wxqw7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" (UID: \"d87d9cce-2585-4025-b0f6-71bf6228012b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:19.688008 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.687973    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:19.965251 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.965128    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m" podStartSLOduration=2.955869597 podStartE2EDuration="5.965106493s" totalImagesPullingTime="3.009236896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:33:19.611186654 +0000 UTC m=+467.423098098" watchObservedRunningTime="2026-09-15 12:33:19.965106493 +0000 UTC m=+467.777017936"
Sep 15 12:33:19.968548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:19.967250    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"]
Sep 15 12:33:19.972557 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:33:19.972530    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd87d9cce_2585_4025_b0f6_71bf6228012b.slice/crio-f99cd3222d67fe3c66f224a5e5bf4be8a3c4576b8c8f71e9941c7de9dffa0a2f WatchSource:0}: Error finding container f99cd3222d67fe3c66f224a5e5bf4be8a3c4576b8c8f71e9941c7de9dffa0a2f: Status 404 returned error can't find the container with id f99cd3222d67fe3c66f224a5e5bf4be8a3c4576b8c8f71e9941c7de9dffa0a2f
Sep 15 12:33:20.579871 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:20.579835    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt" event={"ID":"d87d9cce-2585-4025-b0f6-71bf6228012b","Type":"ContainerStarted","Data":"f99cd3222d67fe3c66f224a5e5bf4be8a3c4576b8c8f71e9941c7de9dffa0a2f"}
Sep 15 12:33:21.330192 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:21.330154    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"]
Sep 15 12:33:21.333354 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:21.333331    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:21.355148 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:21.355117    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kkxjw\""
Sep 15 12:33:21.360685 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:21.360619    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"]
Sep 15 12:33:21.461799 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:21.461738    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jmm5\" (UniqueName: \"kubernetes.io/projected/0797e654-7a72-4639-bd3a-8aef5ffcdc40-kube-api-access-4jmm5\") pod \"limitador-operator-controller-manager-74495c69f8-5vm9x\" (UID: \"0797e654-7a72-4639-bd3a-8aef5ffcdc40\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:21.563123 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:21.563064    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jmm5\" (UniqueName: \"kubernetes.io/projected/0797e654-7a72-4639-bd3a-8aef5ffcdc40-kube-api-access-4jmm5\") pod \"limitador-operator-controller-manager-74495c69f8-5vm9x\" (UID: \"0797e654-7a72-4639-bd3a-8aef5ffcdc40\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:21.578068 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:21.578033    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jmm5\" (UniqueName: \"kubernetes.io/projected/0797e654-7a72-4639-bd3a-8aef5ffcdc40-kube-api-access-4jmm5\") pod \"limitador-operator-controller-manager-74495c69f8-5vm9x\" (UID: \"0797e654-7a72-4639-bd3a-8aef5ffcdc40\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:21.646315 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:21.646238    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:21.929498 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:21.929383    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"]
Sep 15 12:33:21.932077 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:33:21.932048    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0797e654_7a72_4639_bd3a_8aef5ffcdc40.slice/crio-a92ccd0078e446bbbaf472e00a32cc6b7c173d2c18fef41785f9fe40476c44f0 WatchSource:0}: Error finding container a92ccd0078e446bbbaf472e00a32cc6b7c173d2c18fef41785f9fe40476c44f0: Status 404 returned error can't find the container with id a92ccd0078e446bbbaf472e00a32cc6b7c173d2c18fef41785f9fe40476c44f0
Sep 15 12:33:22.592406 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:22.592365    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x" event={"ID":"0797e654-7a72-4639-bd3a-8aef5ffcdc40","Type":"ContainerStarted","Data":"a92ccd0078e446bbbaf472e00a32cc6b7c173d2c18fef41785f9fe40476c44f0"}
Sep 15 12:33:24.605185 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:24.605144    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt" event={"ID":"d87d9cce-2585-4025-b0f6-71bf6228012b","Type":"ContainerStarted","Data":"e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f"}
Sep 15 12:33:24.605573 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:24.605245    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:24.625443 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:24.625348    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt" podStartSLOduration=1.7973957600000001 podStartE2EDuration="5.625334221s" totalImagesPullingTime="3.827938461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:33:24.624242868 +0000 UTC m=+472.436154309" watchObservedRunningTime="2026-09-15 12:33:24.625334221 +0000 UTC m=+472.437245663"
Sep 15 12:33:25.612369 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:25.612326    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x" event={"ID":"0797e654-7a72-4639-bd3a-8aef5ffcdc40","Type":"ContainerStarted","Data":"a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d"}
Sep 15 12:33:25.612904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:25.612659    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:25.630164 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:25.630120    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x" podStartSLOduration=1.94603528 podStartE2EDuration="4.630106347s" totalImagesPullingTime="2.684071067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:33:25.628607534 +0000 UTC m=+473.440518978" watchObservedRunningTime="2026-09-15 12:33:25.630106347 +0000 UTC m=+473.442017788"
Sep 15 12:33:30.582062 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:30.582033    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-snq5m"
Sep 15 12:33:30.582426 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:30.582181    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-s7dhm"
Sep 15 12:33:35.614707 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:35.614675    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:36.838317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:36.838278    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"]
Sep 15 12:33:36.841637 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:36.841619    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"
Sep 15 12:33:36.853892 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:36.853866    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"]
Sep 15 12:33:36.892151 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:36.892116    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrbhn\" (UniqueName: \"kubernetes.io/projected/f93dac12-0895-42f9-9991-d91283c6d073-kube-api-access-lrbhn\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-75wqn\" (UID: \"f93dac12-0895-42f9-9991-d91283c6d073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"
Sep 15 12:33:36.892355 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:36.892161    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93dac12-0895-42f9-9991-d91283c6d073-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-75wqn\" (UID: \"f93dac12-0895-42f9-9991-d91283c6d073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"
Sep 15 12:33:36.992798 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:36.992745    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrbhn\" (UniqueName: \"kubernetes.io/projected/f93dac12-0895-42f9-9991-d91283c6d073-kube-api-access-lrbhn\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-75wqn\" (UID: \"f93dac12-0895-42f9-9991-d91283c6d073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"
Sep 15 12:33:36.993019 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:36.992808    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93dac12-0895-42f9-9991-d91283c6d073-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-75wqn\" (UID: \"f93dac12-0895-42f9-9991-d91283c6d073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"
Sep 15 12:33:36.993277 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:36.993245    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93dac12-0895-42f9-9991-d91283c6d073-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-75wqn\" (UID: \"f93dac12-0895-42f9-9991-d91283c6d073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"
Sep 15 12:33:37.013496 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.013465    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"]
Sep 15 12:33:37.013764 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:37.013745    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lrbhn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn" podUID="f93dac12-0895-42f9-9991-d91283c6d073"
Sep 15 12:33:37.014232 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.014206    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrbhn\" (UniqueName: \"kubernetes.io/projected/f93dac12-0895-42f9-9991-d91283c6d073-kube-api-access-lrbhn\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-75wqn\" (UID: \"f93dac12-0895-42f9-9991-d91283c6d073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"
Sep 15 12:33:37.056206 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.056163    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"]
Sep 15 12:33:37.059608 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.059585    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:37.075568 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.075540    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"]
Sep 15 12:33:37.094255 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.094168    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/becc2970-f740-4e4e-868a-2cdb8aa0ef14-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-tlvpz\" (UID: \"becc2970-f740-4e4e-868a-2cdb8aa0ef14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:37.094255 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.094226    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8ngd\" (UniqueName: \"kubernetes.io/projected/becc2970-f740-4e4e-868a-2cdb8aa0ef14-kube-api-access-v8ngd\") pod \"kuadrant-operator-controller-manager-5f685c496f-tlvpz\" (UID: \"becc2970-f740-4e4e-868a-2cdb8aa0ef14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:37.195399 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.195355    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ngd\" (UniqueName: \"kubernetes.io/projected/becc2970-f740-4e4e-868a-2cdb8aa0ef14-kube-api-access-v8ngd\") pod \"kuadrant-operator-controller-manager-5f685c496f-tlvpz\" (UID: \"becc2970-f740-4e4e-868a-2cdb8aa0ef14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:37.195577 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.195459    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/becc2970-f740-4e4e-868a-2cdb8aa0ef14-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-tlvpz\" (UID: \"becc2970-f740-4e4e-868a-2cdb8aa0ef14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:37.195859 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.195842    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/becc2970-f740-4e4e-868a-2cdb8aa0ef14-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-tlvpz\" (UID: \"becc2970-f740-4e4e-868a-2cdb8aa0ef14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:37.203349 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.203327    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8ngd\" (UniqueName: \"kubernetes.io/projected/becc2970-f740-4e4e-868a-2cdb8aa0ef14-kube-api-access-v8ngd\") pod \"kuadrant-operator-controller-manager-5f685c496f-tlvpz\" (UID: \"becc2970-f740-4e4e-868a-2cdb8aa0ef14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:37.370202 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.370106    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:37.623458 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.623377    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:37.646295 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.646258    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"]
Sep 15 12:33:37.650293 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:33:37.650248    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbecc2970_f740_4e4e_868a_2cdb8aa0ef14.slice/crio-169d50e228df55cefedf894d98d456c1c10ca38f4ee36536d3482e49fc46ac57 WatchSource:0}: Error finding container 169d50e228df55cefedf894d98d456c1c10ca38f4ee36536d3482e49fc46ac57: Status 404 returned error can't find the container with id 169d50e228df55cefedf894d98d456c1c10ca38f4ee36536d3482e49fc46ac57
Sep 15 12:33:37.667843 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.667809    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz" event={"ID":"becc2970-f740-4e4e-868a-2cdb8aa0ef14","Type":"ContainerStarted","Data":"169d50e228df55cefedf894d98d456c1c10ca38f4ee36536d3482e49fc46ac57"}
Sep 15 12:33:37.667952 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.667869    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"
Sep 15 12:33:37.676731 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.676708    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"
Sep 15 12:33:37.698691 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.698663    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93dac12-0895-42f9-9991-d91283c6d073-extensions-socket-volume\") pod \"f93dac12-0895-42f9-9991-d91283c6d073\" (UID: \"f93dac12-0895-42f9-9991-d91283c6d073\") "
Sep 15 12:33:37.698889 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.698758    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrbhn\" (UniqueName: \"kubernetes.io/projected/f93dac12-0895-42f9-9991-d91283c6d073-kube-api-access-lrbhn\") pod \"f93dac12-0895-42f9-9991-d91283c6d073\" (UID: \"f93dac12-0895-42f9-9991-d91283c6d073\") "
Sep 15 12:33:37.699019 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.698996    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f93dac12-0895-42f9-9991-d91283c6d073-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f93dac12-0895-42f9-9991-d91283c6d073" (UID: "f93dac12-0895-42f9-9991-d91283c6d073"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:33:37.699087 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.699066    2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93dac12-0895-42f9-9991-d91283c6d073-extensions-socket-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:33:37.701078 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.701053    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f93dac12-0895-42f9-9991-d91283c6d073-kube-api-access-lrbhn" (OuterVolumeSpecName: "kube-api-access-lrbhn") pod "f93dac12-0895-42f9-9991-d91283c6d073" (UID: "f93dac12-0895-42f9-9991-d91283c6d073"). InnerVolumeSpecName "kube-api-access-lrbhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:33:37.743697 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.743663    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"]
Sep 15 12:33:37.749156 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.749121    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"]
Sep 15 12:33:37.757800 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.757761    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"]
Sep 15 12:33:37.768134 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.768106    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"]
Sep 15 12:33:37.768443 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.768431    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="becc2970-f740-4e4e-868a-2cdb8aa0ef14" containerName="manager"
Sep 15 12:33:37.768483 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.768446    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="becc2970-f740-4e4e-868a-2cdb8aa0ef14" containerName="manager"
Sep 15 12:33:37.768522 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.768513    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="becc2970-f740-4e4e-868a-2cdb8aa0ef14" containerName="manager"
Sep 15 12:33:37.771846 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.771823    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:33:37.772023 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.771929    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"]
Sep 15 12:33:37.783716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.783689    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"]
Sep 15 12:33:37.783928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.783906    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" containerName="manager" containerID="cri-o://e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f" gracePeriod=2
Sep 15 12:33:37.785557 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.785532    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"]
Sep 15 12:33:37.792981 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.792945    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"]
Sep 15 12:33:37.809802 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.808214    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn7cx\" (UniqueName: \"kubernetes.io/projected/71ae7116-e82e-40c4-9c98-d0b5276ab746-kube-api-access-sn7cx\") pod \"kuadrant-operator-controller-manager-5f685c496f-hc8pd\" (UID: \"71ae7116-e82e-40c4-9c98-d0b5276ab746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:33:37.809802 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.808308    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71ae7116-e82e-40c4-9c98-d0b5276ab746-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-hc8pd\" (UID: \"71ae7116-e82e-40c4-9c98-d0b5276ab746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:33:37.809802 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.808413    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrbhn\" (UniqueName: \"kubernetes.io/projected/f93dac12-0895-42f9-9991-d91283c6d073-kube-api-access-lrbhn\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:33:37.817803 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.815389    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"]
Sep 15 12:33:37.817803 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:37.815908    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:37.817803 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.815922    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" containerName="manager"
Sep 15 12:33:37.817803 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.815950    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" containerName="manager"
Sep 15 12:33:37.817803 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.816037    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" containerName="manager"
Sep 15 12:33:37.820466 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.820441    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"]
Sep 15 12:33:37.820741 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.820596    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:37.820875 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.820817    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x" podUID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" containerName="manager" containerID="cri-o://a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d" gracePeriod=2
Sep 15 12:33:37.821720 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.821693    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"]
Sep 15 12:33:37.824387 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:37.824353    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5vm9x\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:37.826605 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:37.826577    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:37.828868 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:37.828844    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:37.831186 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:37.831098    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5vm9x\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:37.834468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.834436    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"]
Sep 15 12:33:37.838587 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.838564    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh"]
Sep 15 12:33:37.839103 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.839081    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" containerName="manager"
Sep 15 12:33:37.839268 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.839255    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" containerName="manager"
Sep 15 12:33:37.839537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.839511    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" containerName="manager"
Sep 15 12:33:37.845131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.845109    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh"
Sep 15 12:33:37.850727 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.850700    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh"]
Sep 15 12:33:37.862950 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:37.862901    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:37.882962 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:37.882868    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5vm9x\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:37.908917 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.908878    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpbfg\" (UniqueName: \"kubernetes.io/projected/d359578d-e21f-4300-9468-b0e831709e80-kube-api-access-bpbfg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qgngw\" (UID: \"d359578d-e21f-4300-9468-b0e831709e80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:37.909055 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.908970    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn7cx\" (UniqueName: \"kubernetes.io/projected/71ae7116-e82e-40c4-9c98-d0b5276ab746-kube-api-access-sn7cx\") pod \"kuadrant-operator-controller-manager-5f685c496f-hc8pd\" (UID: \"71ae7116-e82e-40c4-9c98-d0b5276ab746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:33:37.909111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.909052    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf5j2\" (UniqueName: \"kubernetes.io/projected/e29364af-ff4f-4457-9dc0-dec168150c64-kube-api-access-wf5j2\") pod \"limitador-operator-controller-manager-74495c69f8-jw4bh\" (UID: \"e29364af-ff4f-4457-9dc0-dec168150c64\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh"
Sep 15 12:33:37.909111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.909087    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71ae7116-e82e-40c4-9c98-d0b5276ab746-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-hc8pd\" (UID: \"71ae7116-e82e-40c4-9c98-d0b5276ab746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:33:37.909185 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.909116    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d359578d-e21f-4300-9468-b0e831709e80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qgngw\" (UID: \"d359578d-e21f-4300-9468-b0e831709e80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:37.909422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.909406    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71ae7116-e82e-40c4-9c98-d0b5276ab746-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-hc8pd\" (UID: \"71ae7116-e82e-40c4-9c98-d0b5276ab746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:33:37.922692 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:37.922664    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn7cx\" (UniqueName: \"kubernetes.io/projected/71ae7116-e82e-40c4-9c98-d0b5276ab746-kube-api-access-sn7cx\") pod \"kuadrant-operator-controller-manager-5f685c496f-hc8pd\" (UID: \"71ae7116-e82e-40c4-9c98-d0b5276ab746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:33:38.009636 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.009601    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpbfg\" (UniqueName: \"kubernetes.io/projected/d359578d-e21f-4300-9468-b0e831709e80-kube-api-access-bpbfg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qgngw\" (UID: \"d359578d-e21f-4300-9468-b0e831709e80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:38.009832 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.009702    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf5j2\" (UniqueName: \"kubernetes.io/projected/e29364af-ff4f-4457-9dc0-dec168150c64-kube-api-access-wf5j2\") pod \"limitador-operator-controller-manager-74495c69f8-jw4bh\" (UID: \"e29364af-ff4f-4457-9dc0-dec168150c64\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh"
Sep 15 12:33:38.009832 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.009733    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d359578d-e21f-4300-9468-b0e831709e80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qgngw\" (UID: \"d359578d-e21f-4300-9468-b0e831709e80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:38.010140 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.010118    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d359578d-e21f-4300-9468-b0e831709e80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qgngw\" (UID: \"d359578d-e21f-4300-9468-b0e831709e80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:38.018765 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.018738    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpbfg\" (UniqueName: \"kubernetes.io/projected/d359578d-e21f-4300-9468-b0e831709e80-kube-api-access-bpbfg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qgngw\" (UID: \"d359578d-e21f-4300-9468-b0e831709e80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:38.019670 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.019647    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf5j2\" (UniqueName: \"kubernetes.io/projected/e29364af-ff4f-4457-9dc0-dec168150c64-kube-api-access-wf5j2\") pod \"limitador-operator-controller-manager-74495c69f8-jw4bh\" (UID: \"e29364af-ff4f-4457-9dc0-dec168150c64\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh"
Sep 15 12:33:38.056235 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.056212    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:38.058415 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:38.058387    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:38.059063 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.059048    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:38.060287 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:38.060263    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5vm9x\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:38.062136 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:38.062108    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xkt\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:38.063905 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:38.063884    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5vm9x\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:38.084126 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.084101    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:33:38.110738 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.110704    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d87d9cce-2585-4025-b0f6-71bf6228012b-extensions-socket-volume\") pod \"d87d9cce-2585-4025-b0f6-71bf6228012b\" (UID: \"d87d9cce-2585-4025-b0f6-71bf6228012b\") "
Sep 15 12:33:38.110883 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.110751    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxqw7\" (UniqueName: \"kubernetes.io/projected/d87d9cce-2585-4025-b0f6-71bf6228012b-kube-api-access-wxqw7\") pod \"d87d9cce-2585-4025-b0f6-71bf6228012b\" (UID: \"d87d9cce-2585-4025-b0f6-71bf6228012b\") "
Sep 15 12:33:38.110883 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.110815    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jmm5\" (UniqueName: \"kubernetes.io/projected/0797e654-7a72-4639-bd3a-8aef5ffcdc40-kube-api-access-4jmm5\") pod \"0797e654-7a72-4639-bd3a-8aef5ffcdc40\" (UID: \"0797e654-7a72-4639-bd3a-8aef5ffcdc40\") "
Sep 15 12:33:38.111116 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.111092    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d87d9cce-2585-4025-b0f6-71bf6228012b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d87d9cce-2585-4025-b0f6-71bf6228012b" (UID: "d87d9cce-2585-4025-b0f6-71bf6228012b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:33:38.113066 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.113032    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d87d9cce-2585-4025-b0f6-71bf6228012b-kube-api-access-wxqw7" (OuterVolumeSpecName: "kube-api-access-wxqw7") pod "d87d9cce-2585-4025-b0f6-71bf6228012b" (UID: "d87d9cce-2585-4025-b0f6-71bf6228012b"). InnerVolumeSpecName "kube-api-access-wxqw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:33:38.113164 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.113073    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0797e654-7a72-4639-bd3a-8aef5ffcdc40-kube-api-access-4jmm5" (OuterVolumeSpecName: "kube-api-access-4jmm5") pod "0797e654-7a72-4639-bd3a-8aef5ffcdc40" (UID: "0797e654-7a72-4639-bd3a-8aef5ffcdc40"). InnerVolumeSpecName "kube-api-access-4jmm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:33:38.204895 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.204857    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:38.211055 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.211033    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh"
Sep 15 12:33:38.212394 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.212366    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jmm5\" (UniqueName: \"kubernetes.io/projected/0797e654-7a72-4639-bd3a-8aef5ffcdc40-kube-api-access-4jmm5\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:33:38.212394 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.212389    2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d87d9cce-2585-4025-b0f6-71bf6228012b-extensions-socket-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:33:38.212527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.212400    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxqw7\" (UniqueName: \"kubernetes.io/projected/d87d9cce-2585-4025-b0f6-71bf6228012b-kube-api-access-wxqw7\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:33:38.361230 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.361198    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"]
Sep 15 12:33:38.364724 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:33:38.364693    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ae7116_e82e_40c4_9c98_d0b5276ab746.slice/crio-faf46487d9dd80d58af8abb37248486c9d9617ec8ddc8427c1d617e13f9c61bd WatchSource:0}: Error finding container faf46487d9dd80d58af8abb37248486c9d9617ec8ddc8427c1d617e13f9c61bd: Status 404 returned error can't find the container with id faf46487d9dd80d58af8abb37248486c9d9617ec8ddc8427c1d617e13f9c61bd
Sep 15 12:33:38.483954 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.483921    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"]
Sep 15 12:33:38.486657 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:33:38.486629    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd359578d_e21f_4300_9468_b0e831709e80.slice/crio-a69283888403d792f67e291792e8b6617ea0fb7df075dbf384ffa31a034a2901 WatchSource:0}: Error finding container a69283888403d792f67e291792e8b6617ea0fb7df075dbf384ffa31a034a2901: Status 404 returned error can't find the container with id a69283888403d792f67e291792e8b6617ea0fb7df075dbf384ffa31a034a2901
Sep 15 12:33:38.496396 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.496370    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh"]
Sep 15 12:33:38.500506 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:33:38.500476    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29364af_ff4f_4457_9dc0_dec168150c64.slice/crio-ebf06eab310efd80ce3abf4f579c99e2b0f5b62e67b84cc55d36eae0a8d188eb WatchSource:0}: Error finding container ebf06eab310efd80ce3abf4f579c99e2b0f5b62e67b84cc55d36eae0a8d188eb: Status 404 returned error can't find the container with id ebf06eab310efd80ce3abf4f579c99e2b0f5b62e67b84cc55d36eae0a8d188eb
Sep 15 12:33:38.652051 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.651972    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" path="/var/lib/kubelet/pods/0797e654-7a72-4639-bd3a-8aef5ffcdc40/volumes"
Sep 15 12:33:38.652379 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.652364    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d87d9cce-2585-4025-b0f6-71bf6228012b" path="/var/lib/kubelet/pods/d87d9cce-2585-4025-b0f6-71bf6228012b/volumes"
Sep 15 12:33:38.652684 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.652672    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f93dac12-0895-42f9-9991-d91283c6d073" path="/var/lib/kubelet/pods/f93dac12-0895-42f9-9991-d91283c6d073/volumes"
Sep 15 12:33:38.674529 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.674500    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw" event={"ID":"d359578d-e21f-4300-9468-b0e831709e80","Type":"ContainerStarted","Data":"1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea"}
Sep 15 12:33:38.674637 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.674560    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:38.674637 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.674583    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw" event={"ID":"d359578d-e21f-4300-9468-b0e831709e80","Type":"ContainerStarted","Data":"a69283888403d792f67e291792e8b6617ea0fb7df075dbf384ffa31a034a2901"}
Sep 15 12:33:38.676191 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.676167    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh" event={"ID":"e29364af-ff4f-4457-9dc0-dec168150c64","Type":"ContainerStarted","Data":"19e6c26314daa60a985924e8cb3d03e504aeaeac5f48ffc77291315befd75a58"}
Sep 15 12:33:38.676279 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.676213    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh"
Sep 15 12:33:38.676279 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.676230    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh" event={"ID":"e29364af-ff4f-4457-9dc0-dec168150c64","Type":"ContainerStarted","Data":"ebf06eab310efd80ce3abf4f579c99e2b0f5b62e67b84cc55d36eae0a8d188eb"}
Sep 15 12:33:38.678019 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:38.678001    2586 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz" container="manager"
Sep 15 12:33:38.679592 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.679568    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd" event={"ID":"71ae7116-e82e-40c4-9c98-d0b5276ab746","Type":"ContainerStarted","Data":"ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306"}
Sep 15 12:33:38.679693 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.679615    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd" event={"ID":"71ae7116-e82e-40c4-9c98-d0b5276ab746","Type":"ContainerStarted","Data":"faf46487d9dd80d58af8abb37248486c9d9617ec8ddc8427c1d617e13f9c61bd"}
Sep 15 12:33:38.679693 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.679682    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:33:38.681052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.681032    2586 generic.go:356] "Generic (PLEG): container finished" podID="0797e654-7a72-4639-bd3a-8aef5ffcdc40" containerID="a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d" exitCode=0
Sep 15 12:33:38.681157 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.681095    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5vm9x"
Sep 15 12:33:38.681157 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.681115    2586 scope.go:117] "RemoveContainer" containerID="a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d"
Sep 15 12:33:38.682916 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.682892    2586 generic.go:356] "Generic (PLEG): container finished" podID="d87d9cce-2585-4025-b0f6-71bf6228012b" containerID="e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f" exitCode=0
Sep 15 12:33:38.683109 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.682974    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-75wqn"
Sep 15 12:33:38.683402 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.683281    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xkt"
Sep 15 12:33:38.691793 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.691763    2586 scope.go:117] "RemoveContainer" containerID="a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d"
Sep 15 12:33:38.692075 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:38.692055    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d\": container with ID starting with a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d not found: ID does not exist" containerID="a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d"
Sep 15 12:33:38.692133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.692085    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d"} err="failed to get container status \"a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d\": rpc error: code = NotFound desc = could not find container \"a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d\": container with ID starting with a10f20f04d300c2e49aad4620a8a1115f80904eb04366093dc1f54fb3c96617d not found: ID does not exist"
Sep 15 12:33:38.692133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.692103    2586 scope.go:117] "RemoveContainer" containerID="e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f"
Sep 15 12:33:38.701461 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.701428    2586 scope.go:117] "RemoveContainer" containerID="e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f"
Sep 15 12:33:38.701830 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:38.701808    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f\": container with ID starting with e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f not found: ID does not exist" containerID="e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f"
Sep 15 12:33:38.701929 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.701834    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f"} err="failed to get container status \"e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f\": rpc error: code = NotFound desc = could not find container \"e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f\": container with ID starting with e560bd3dd5304c5a835d6e36729bdbbf4cee292dbc9339260f5dc975452a340f not found: ID does not exist"
Sep 15 12:33:38.713507 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.713452    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw" podStartSLOduration=1.7134337400000001 podStartE2EDuration="1.71343374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:33:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:33:38.712125367 +0000 UTC m=+486.524036812" watchObservedRunningTime="2026-09-15 12:33:38.71343374 +0000 UTC m=+486.525345182"
Sep 15 12:33:38.734004 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.733958    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh" podStartSLOduration=1.733942727 podStartE2EDuration="1.733942727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:33:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:33:38.733339541 +0000 UTC m=+486.545250996" watchObservedRunningTime="2026-09-15 12:33:38.733942727 +0000 UTC m=+486.545854168"
Sep 15 12:33:38.753120 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:38.753060    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd" podStartSLOduration=1.753042955 podStartE2EDuration="1.753042955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:33:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:33:38.752937215 +0000 UTC m=+486.564848657" watchObservedRunningTime="2026-09-15 12:33:38.753042955 +0000 UTC m=+486.564954399"
Sep 15 12:33:39.693021 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:39.692977    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz" podUID="becc2970-f740-4e4e-868a-2cdb8aa0ef14" containerName="manager" containerID="cri-o://4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b" gracePeriod=2
Sep 15 12:33:39.695416 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:39.695359    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-tlvpz\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="becc2970-f740-4e4e-868a-2cdb8aa0ef14" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:39.933659 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:39.933632    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:39.935739 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:39.935717    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-tlvpz\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="becc2970-f740-4e4e-868a-2cdb8aa0ef14" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:40.027547 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.027515    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/becc2970-f740-4e4e-868a-2cdb8aa0ef14-extensions-socket-volume\") pod \"becc2970-f740-4e4e-868a-2cdb8aa0ef14\" (UID: \"becc2970-f740-4e4e-868a-2cdb8aa0ef14\") "
Sep 15 12:33:40.027711 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.027561    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8ngd\" (UniqueName: \"kubernetes.io/projected/becc2970-f740-4e4e-868a-2cdb8aa0ef14-kube-api-access-v8ngd\") pod \"becc2970-f740-4e4e-868a-2cdb8aa0ef14\" (UID: \"becc2970-f740-4e4e-868a-2cdb8aa0ef14\") "
Sep 15 12:33:40.027823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.027793    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/becc2970-f740-4e4e-868a-2cdb8aa0ef14-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "becc2970-f740-4e4e-868a-2cdb8aa0ef14" (UID: "becc2970-f740-4e4e-868a-2cdb8aa0ef14"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:33:40.029596 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.029574    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/becc2970-f740-4e4e-868a-2cdb8aa0ef14-kube-api-access-v8ngd" (OuterVolumeSpecName: "kube-api-access-v8ngd") pod "becc2970-f740-4e4e-868a-2cdb8aa0ef14" (UID: "becc2970-f740-4e4e-868a-2cdb8aa0ef14"). InnerVolumeSpecName "kube-api-access-v8ngd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:33:40.128880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.128833    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8ngd\" (UniqueName: \"kubernetes.io/projected/becc2970-f740-4e4e-868a-2cdb8aa0ef14-kube-api-access-v8ngd\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:33:40.128880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.128870    2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/becc2970-f740-4e4e-868a-2cdb8aa0ef14-extensions-socket-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:33:40.651881 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.651846    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="becc2970-f740-4e4e-868a-2cdb8aa0ef14" path="/var/lib/kubelet/pods/becc2970-f740-4e4e-868a-2cdb8aa0ef14/volumes"
Sep 15 12:33:40.701080 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.701052    2586 generic.go:356] "Generic (PLEG): container finished" podID="becc2970-f740-4e4e-868a-2cdb8aa0ef14" containerID="4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b" exitCode=2
Sep 15 12:33:40.701498 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.701094    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:40.701498 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.701149    2586 scope.go:117] "RemoveContainer" containerID="4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b"
Sep 15 12:33:40.705278 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:40.705250    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-tlvpz\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="becc2970-f740-4e4e-868a-2cdb8aa0ef14" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:40.709263 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.709246    2586 scope.go:117] "RemoveContainer" containerID="4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b"
Sep 15 12:33:40.709558 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:40.709539    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b\": container with ID starting with 4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b not found: ID does not exist" containerID="4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b"
Sep 15 12:33:40.709602 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:40.709566    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b"} err="failed to get container status \"4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b\": rpc error: code = NotFound desc = could not find container \"4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b\": container with ID starting with 4efb23a82479e6f41033171bf91160db72f47808f949ccdd1c0cca547a67208b not found: ID does not exist"
Sep 15 12:33:42.650870 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:42.650832    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-tlvpz\" is forbidden: User \"system:node:ip-10-0-141-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-20.ec2.internal' and this object" podUID="becc2970-f740-4e4e-868a-2cdb8aa0ef14" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tlvpz"
Sep 15 12:33:49.698133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:49.698092    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jw4bh"
Sep 15 12:33:49.698531 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:49.698177    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:49.698531 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:49.698347    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:33:49.784959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:49.784919    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"]
Sep 15 12:33:49.785140 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:49.785117    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw" podUID="d359578d-e21f-4300-9468-b0e831709e80" containerName="manager" containerID="cri-o://1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea" gracePeriod=10
Sep 15 12:33:50.044211 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.044177    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:50.121655 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.121621    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d359578d-e21f-4300-9468-b0e831709e80-extensions-socket-volume\") pod \"d359578d-e21f-4300-9468-b0e831709e80\" (UID: \"d359578d-e21f-4300-9468-b0e831709e80\") "
Sep 15 12:33:50.121655 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.121661    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpbfg\" (UniqueName: \"kubernetes.io/projected/d359578d-e21f-4300-9468-b0e831709e80-kube-api-access-bpbfg\") pod \"d359578d-e21f-4300-9468-b0e831709e80\" (UID: \"d359578d-e21f-4300-9468-b0e831709e80\") "
Sep 15 12:33:50.122066 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.122037    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d359578d-e21f-4300-9468-b0e831709e80-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d359578d-e21f-4300-9468-b0e831709e80" (UID: "d359578d-e21f-4300-9468-b0e831709e80"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:33:50.123864 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.123839    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d359578d-e21f-4300-9468-b0e831709e80-kube-api-access-bpbfg" (OuterVolumeSpecName: "kube-api-access-bpbfg") pod "d359578d-e21f-4300-9468-b0e831709e80" (UID: "d359578d-e21f-4300-9468-b0e831709e80"). InnerVolumeSpecName "kube-api-access-bpbfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:33:50.152841 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.152808    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"]
Sep 15 12:33:50.153285 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.153267    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d359578d-e21f-4300-9468-b0e831709e80" containerName="manager"
Sep 15 12:33:50.153335 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.153290    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="d359578d-e21f-4300-9468-b0e831709e80" containerName="manager"
Sep 15 12:33:50.153395 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.153383    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="d359578d-e21f-4300-9468-b0e831709e80" containerName="manager"
Sep 15 12:33:50.156254 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.156240    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 12:33:50.172808 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.172768    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"]
Sep 15 12:33:50.223012 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.222909    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qpxcx\" (UID: \"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 12:33:50.223012 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.222958    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5jcg\" (UniqueName: \"kubernetes.io/projected/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-kube-api-access-g5jcg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qpxcx\" (UID: \"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 12:33:50.223201 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.223071    2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d359578d-e21f-4300-9468-b0e831709e80-extensions-socket-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:33:50.223201 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.223094    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpbfg\" (UniqueName: \"kubernetes.io/projected/d359578d-e21f-4300-9468-b0e831709e80-kube-api-access-bpbfg\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:33:50.323843 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.323800    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qpxcx\" (UID: \"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 12:33:50.324029 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.323848    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5jcg\" (UniqueName: \"kubernetes.io/projected/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-kube-api-access-g5jcg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qpxcx\" (UID: \"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 12:33:50.324182 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.324161    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qpxcx\" (UID: \"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 12:33:50.358772 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.358735    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5jcg\" (UniqueName: \"kubernetes.io/projected/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-kube-api-access-g5jcg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qpxcx\" (UID: \"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 12:33:50.465883 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.465840    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 12:33:50.741083 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.741059    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"]
Sep 15 12:33:50.746411 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:33:50.746376    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11269c9_c6b9_40b9_b73f_c964ef8ee7e7.slice/crio-4ef56e78023aa4287b954f59030feb307d8217c3c9a033748096f09944a8d5d9 WatchSource:0}: Error finding container 4ef56e78023aa4287b954f59030feb307d8217c3c9a033748096f09944a8d5d9: Status 404 returned error can't find the container with id 4ef56e78023aa4287b954f59030feb307d8217c3c9a033748096f09944a8d5d9
Sep 15 12:33:50.750868 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.750845    2586 generic.go:356] "Generic (PLEG): container finished" podID="d359578d-e21f-4300-9468-b0e831709e80" containerID="1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea" exitCode=0
Sep 15 12:33:50.750965 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.750883    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"
Sep 15 12:33:50.750965 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.750927    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw" event={"ID":"d359578d-e21f-4300-9468-b0e831709e80","Type":"ContainerDied","Data":"1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea"}
Sep 15 12:33:50.751079 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.750987    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw" event={"ID":"d359578d-e21f-4300-9468-b0e831709e80","Type":"ContainerDied","Data":"a69283888403d792f67e291792e8b6617ea0fb7df075dbf384ffa31a034a2901"}
Sep 15 12:33:50.751079 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.751020    2586 scope.go:117] "RemoveContainer" containerID="1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea"
Sep 15 12:33:50.761609 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.761593    2586 scope.go:117] "RemoveContainer" containerID="1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea"
Sep 15 12:33:50.761891 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:33:50.761871    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea\": container with ID starting with 1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea not found: ID does not exist" containerID="1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea"
Sep 15 12:33:50.761959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.761899    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea"} err="failed to get container status \"1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea\": rpc error: code = NotFound desc = could not find container \"1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea\": container with ID starting with 1238284b5eff7bf8f07ed351f1d55ff9c0a48f9a8630d970ea978f966bd283ea not found: ID does not exist"
Sep 15 12:33:50.770791 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.770752    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"]
Sep 15 12:33:50.780691 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:50.780668    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qgngw"]
Sep 15 12:33:51.757106 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:51.757069    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx" event={"ID":"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7","Type":"ContainerStarted","Data":"f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67"}
Sep 15 12:33:51.757577 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:51.757163    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx" event={"ID":"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7","Type":"ContainerStarted","Data":"4ef56e78023aa4287b954f59030feb307d8217c3c9a033748096f09944a8d5d9"}
Sep 15 12:33:51.757577 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:51.757195    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 12:33:51.782225 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:51.782170    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx" podStartSLOduration=1.782153487 podStartE2EDuration="1.782153487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:33:51.780544098 +0000 UTC m=+499.592455579" watchObservedRunningTime="2026-09-15 12:33:51.782153487 +0000 UTC m=+499.594064929"
Sep 15 12:33:52.652707 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:33:52.652671    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d359578d-e21f-4300-9468-b0e831709e80" path="/var/lib/kubelet/pods/d359578d-e21f-4300-9468-b0e831709e80/volumes"
Sep 15 12:34:02.766262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:02.766236    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 12:34:02.814332 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:02.814299    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"]
Sep 15 12:34:02.814556 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:02.814532    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd" podUID="71ae7116-e82e-40c4-9c98-d0b5276ab746" containerName="manager" containerID="cri-o://ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306" gracePeriod=10
Sep 15 12:34:03.052295 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.052269    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:34:03.241818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.241768    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71ae7116-e82e-40c4-9c98-d0b5276ab746-extensions-socket-volume\") pod \"71ae7116-e82e-40c4-9c98-d0b5276ab746\" (UID: \"71ae7116-e82e-40c4-9c98-d0b5276ab746\") "
Sep 15 12:34:03.241818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.241821    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn7cx\" (UniqueName: \"kubernetes.io/projected/71ae7116-e82e-40c4-9c98-d0b5276ab746-kube-api-access-sn7cx\") pod \"71ae7116-e82e-40c4-9c98-d0b5276ab746\" (UID: \"71ae7116-e82e-40c4-9c98-d0b5276ab746\") "
Sep 15 12:34:03.242193 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.242164    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71ae7116-e82e-40c4-9c98-d0b5276ab746-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "71ae7116-e82e-40c4-9c98-d0b5276ab746" (UID: "71ae7116-e82e-40c4-9c98-d0b5276ab746"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:34:03.243855 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.243834    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71ae7116-e82e-40c4-9c98-d0b5276ab746-kube-api-access-sn7cx" (OuterVolumeSpecName: "kube-api-access-sn7cx") pod "71ae7116-e82e-40c4-9c98-d0b5276ab746" (UID: "71ae7116-e82e-40c4-9c98-d0b5276ab746"). InnerVolumeSpecName "kube-api-access-sn7cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:34:03.342935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.342856    2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71ae7116-e82e-40c4-9c98-d0b5276ab746-extensions-socket-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:34:03.342935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.342888    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn7cx\" (UniqueName: \"kubernetes.io/projected/71ae7116-e82e-40c4-9c98-d0b5276ab746-kube-api-access-sn7cx\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:34:03.826444 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.826411    2586 generic.go:356] "Generic (PLEG): container finished" podID="71ae7116-e82e-40c4-9c98-d0b5276ab746" containerID="ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306" exitCode=0
Sep 15 12:34:03.826870 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.826451    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd" event={"ID":"71ae7116-e82e-40c4-9c98-d0b5276ab746","Type":"ContainerDied","Data":"ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306"}
Sep 15 12:34:03.826870 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.826481    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"
Sep 15 12:34:03.826870 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.826513    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd" event={"ID":"71ae7116-e82e-40c4-9c98-d0b5276ab746","Type":"ContainerDied","Data":"faf46487d9dd80d58af8abb37248486c9d9617ec8ddc8427c1d617e13f9c61bd"}
Sep 15 12:34:03.826870 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.826535    2586 scope.go:117] "RemoveContainer" containerID="ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306"
Sep 15 12:34:03.835199 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.835180    2586 scope.go:117] "RemoveContainer" containerID="ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306"
Sep 15 12:34:03.835424 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:34:03.835408    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306\": container with ID starting with ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306 not found: ID does not exist" containerID="ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306"
Sep 15 12:34:03.835470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.835433    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306"} err="failed to get container status \"ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306\": rpc error: code = NotFound desc = could not find container \"ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306\": container with ID starting with ca78891a4c463e551549896ad08ac9a7de552a5f15cdeb06d091f3deeea21306 not found: ID does not exist"
Sep 15 12:34:03.851806 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:34:03.851757    2586 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ae7116_e82e_40c4_9c98_d0b5276ab746.slice\": RecentStats: unable to find data in memory cache]"
Sep 15 12:34:03.852151 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.852134    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"]
Sep 15 12:34:03.854585 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:03.854563    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hc8pd"]
Sep 15 12:34:04.652067 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:04.652037    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71ae7116-e82e-40c4-9c98-d0b5276ab746" path="/var/lib/kubelet/pods/71ae7116-e82e-40c4-9c98-d0b5276ab746/volumes"
Sep 15 12:34:06.067794 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.067750    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"]
Sep 15 12:34:06.068260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.068241    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ae7116-e82e-40c4-9c98-d0b5276ab746" containerName="manager"
Sep 15 12:34:06.068347 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.068263    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ae7116-e82e-40c4-9c98-d0b5276ab746" containerName="manager"
Sep 15 12:34:06.068400 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.068349    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="71ae7116-e82e-40c4-9c98-d0b5276ab746" containerName="manager"
Sep 15 12:34:06.071501 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.071480    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.073869 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.073851    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9vj9r\""
Sep 15 12:34:06.073970 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.073887    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 15 12:34:06.083931 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.083908    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"]
Sep 15 12:34:06.263741 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.263705    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.263741 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.263738    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.264035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.263771    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56m82\" (UniqueName: \"kubernetes.io/projected/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-kube-api-access-56m82\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.264035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.263860    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.264035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.263895    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.264035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.263919    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.264035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.263971    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.264035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.264006    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.264240 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.264040    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365028    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365064    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365312 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365189    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365312 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365228    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365312 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365282    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56m82\" (UniqueName: \"kubernetes.io/projected/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-kube-api-access-56m82\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365454 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365311    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365454 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365391    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365454 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365428    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365454 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365445    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365495    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365497    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365755 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365719    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.365880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.365858    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.366135 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.366119    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.367373 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.367357    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.367666 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.367645    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.372588 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.372561    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.372718 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.372699    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56m82\" (UniqueName: \"kubernetes.io/projected/d26fc663-72b4-4ff5-9e56-3d5eb6603e29-kube-api-access-56m82\") pod \"maas-default-gateway-openshift-default-5ccb968c66-dtgvl\" (UID: \"d26fc663-72b4-4ff5-9e56-3d5eb6603e29\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.384541 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.384509    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:06.656566 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.656544    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"]
Sep 15 12:34:06.658650 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:34:06.658621    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd26fc663_72b4_4ff5_9e56_3d5eb6603e29.slice/crio-c472634dae2e80daf6857fd5a0c713aa9e6a3cec7606aa8160881a5c5d55257a WatchSource:0}: Error finding container c472634dae2e80daf6857fd5a0c713aa9e6a3cec7606aa8160881a5c5d55257a: Status 404 returned error can't find the container with id c472634dae2e80daf6857fd5a0c713aa9e6a3cec7606aa8160881a5c5d55257a
Sep 15 12:34:06.844462 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:06.844423    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl" event={"ID":"d26fc663-72b4-4ff5-9e56-3d5eb6603e29","Type":"ContainerStarted","Data":"c472634dae2e80daf6857fd5a0c713aa9e6a3cec7606aa8160881a5c5d55257a"}
Sep 15 12:34:09.375835 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:09.375801    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:34:09.376073 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:09.375878    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:34:09.376073 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:09.375908    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:34:09.861669 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:09.861633    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl" event={"ID":"d26fc663-72b4-4ff5-9e56-3d5eb6603e29","Type":"ContainerStarted","Data":"602968c88322e564710f7c09d0befc224a1fca9e44e3f7d9ced149f5f6353685"}
Sep 15 12:34:09.881200 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:09.881152    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl" podStartSLOduration=1.166152244 podStartE2EDuration="3.881136408s" totalImagesPullingTime="2.714984164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:34:09.880053229 +0000 UTC m=+517.691964672" watchObservedRunningTime="2026-09-15 12:34:09.881136408 +0000 UTC m=+517.693047850"
Sep 15 12:34:10.385073 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:10.385038    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:10.389847 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:10.389824    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:10.866475 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:10.866438    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:10.867302 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:10.867283    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:34:22.154396 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.154362    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4bh6c"]
Sep 15 12:34:22.158561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.158545    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:22.161084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.161056    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8mdnh\""
Sep 15 12:34:22.161214 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.161062    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 15 12:34:22.168035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.168012    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4bh6c"]
Sep 15 12:34:22.199368 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.199341    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8xl4\" (UniqueName: \"kubernetes.io/projected/096c778f-6bb8-45fc-b3eb-05bc59efdafc-kube-api-access-n8xl4\") pod \"limitador-limitador-8588749977-4bh6c\" (UID: \"096c778f-6bb8-45fc-b3eb-05bc59efdafc\") " pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:22.199511 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.199375    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/096c778f-6bb8-45fc-b3eb-05bc59efdafc-config-file\") pod \"limitador-limitador-8588749977-4bh6c\" (UID: \"096c778f-6bb8-45fc-b3eb-05bc59efdafc\") " pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:22.252874 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.252843    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4bh6c"]
Sep 15 12:34:22.300804 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.300744    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8xl4\" (UniqueName: \"kubernetes.io/projected/096c778f-6bb8-45fc-b3eb-05bc59efdafc-kube-api-access-n8xl4\") pod \"limitador-limitador-8588749977-4bh6c\" (UID: \"096c778f-6bb8-45fc-b3eb-05bc59efdafc\") " pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:22.300977 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.300820    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/096c778f-6bb8-45fc-b3eb-05bc59efdafc-config-file\") pod \"limitador-limitador-8588749977-4bh6c\" (UID: \"096c778f-6bb8-45fc-b3eb-05bc59efdafc\") " pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:22.301487 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.301469    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/096c778f-6bb8-45fc-b3eb-05bc59efdafc-config-file\") pod \"limitador-limitador-8588749977-4bh6c\" (UID: \"096c778f-6bb8-45fc-b3eb-05bc59efdafc\") " pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:22.308032 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.308013    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8xl4\" (UniqueName: \"kubernetes.io/projected/096c778f-6bb8-45fc-b3eb-05bc59efdafc-kube-api-access-n8xl4\") pod \"limitador-limitador-8588749977-4bh6c\" (UID: \"096c778f-6bb8-45fc-b3eb-05bc59efdafc\") " pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:22.470310 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.470227    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:22.738761 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.738734    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4bh6c"]
Sep 15 12:34:22.929078 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:22.929039    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c" event={"ID":"096c778f-6bb8-45fc-b3eb-05bc59efdafc","Type":"ContainerStarted","Data":"8f5aa6d7d8d38daefa66531d295db1922b5849fc394d7beb5004a63b680102ad"}
Sep 15 12:34:23.084842 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:23.084749    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-l9bj5"]
Sep 15 12:34:23.089297 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:23.089279    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-l9bj5"
Sep 15 12:34:23.091592 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:23.091571    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jpchf\""
Sep 15 12:34:23.100320 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:23.100302    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-l9bj5"]
Sep 15 12:34:23.209326 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:23.209299    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k467b\" (UniqueName: \"kubernetes.io/projected/4ed1a6e4-1188-4a88-a3e4-00b31892fce3-kube-api-access-k467b\") pod \"authorino-59c974bd48-l9bj5\" (UID: \"4ed1a6e4-1188-4a88-a3e4-00b31892fce3\") " pod="kuadrant-system/authorino-59c974bd48-l9bj5"
Sep 15 12:34:23.309910 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:23.309880    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k467b\" (UniqueName: \"kubernetes.io/projected/4ed1a6e4-1188-4a88-a3e4-00b31892fce3-kube-api-access-k467b\") pod \"authorino-59c974bd48-l9bj5\" (UID: \"4ed1a6e4-1188-4a88-a3e4-00b31892fce3\") " pod="kuadrant-system/authorino-59c974bd48-l9bj5"
Sep 15 12:34:23.319300 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:23.319279    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k467b\" (UniqueName: \"kubernetes.io/projected/4ed1a6e4-1188-4a88-a3e4-00b31892fce3-kube-api-access-k467b\") pod \"authorino-59c974bd48-l9bj5\" (UID: \"4ed1a6e4-1188-4a88-a3e4-00b31892fce3\") " pod="kuadrant-system/authorino-59c974bd48-l9bj5"
Sep 15 12:34:23.398452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:23.398383    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-l9bj5"
Sep 15 12:34:23.673274 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:23.673246    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-l9bj5"]
Sep 15 12:34:23.675123 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:34:23.675093    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ed1a6e4_1188_4a88_a3e4_00b31892fce3.slice/crio-467e77f1b6beb4cf8f4362f724f7cfbee4182a9993a7978ecbdea325ddbcd333 WatchSource:0}: Error finding container 467e77f1b6beb4cf8f4362f724f7cfbee4182a9993a7978ecbdea325ddbcd333: Status 404 returned error can't find the container with id 467e77f1b6beb4cf8f4362f724f7cfbee4182a9993a7978ecbdea325ddbcd333
Sep 15 12:34:23.935111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:23.935037    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-l9bj5" event={"ID":"4ed1a6e4-1188-4a88-a3e4-00b31892fce3","Type":"ContainerStarted","Data":"467e77f1b6beb4cf8f4362f724f7cfbee4182a9993a7978ecbdea325ddbcd333"}
Sep 15 12:34:24.944192 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:24.944091    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c" event={"ID":"096c778f-6bb8-45fc-b3eb-05bc59efdafc","Type":"ContainerStarted","Data":"83320d89dcce932a75c72ec2fd54a19e278eb650f2845cea85afc3d17f3d7f94"}
Sep 15 12:34:24.944609 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:24.944195    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:24.960680 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:24.960631    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c" podStartSLOduration=1.556975575 podStartE2EDuration="2.960614897s" totalImagesPullingTime="1.403639322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:34:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:34:24.958793484 +0000 UTC m=+532.770704918" watchObservedRunningTime="2026-09-15 12:34:24.960614897 +0000 UTC m=+532.772526338"
Sep 15 12:34:25.950836 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:25.950797    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-l9bj5" event={"ID":"4ed1a6e4-1188-4a88-a3e4-00b31892fce3","Type":"ContainerStarted","Data":"1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec"}
Sep 15 12:34:25.965201 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:25.965149    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-l9bj5" podStartSLOduration=1.452653096 podStartE2EDuration="2.965134095s" totalImagesPullingTime="1.512480999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:34:25.963964201 +0000 UTC m=+533.775875644" watchObservedRunningTime="2026-09-15 12:34:25.965134095 +0000 UTC m=+533.777045538"
Sep 15 12:34:35.951828 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:35.951794    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:37.494293 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:37.494258    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4bh6c"]
Sep 15 12:34:37.494751 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:37.494479    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c" podUID="096c778f-6bb8-45fc-b3eb-05bc59efdafc" containerName="limitador" containerID="cri-o://83320d89dcce932a75c72ec2fd54a19e278eb650f2845cea85afc3d17f3d7f94" gracePeriod=30
Sep 15 12:34:38.005416 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:38.005382    2586 generic.go:356] "Generic (PLEG): container finished" podID="096c778f-6bb8-45fc-b3eb-05bc59efdafc" containerID="83320d89dcce932a75c72ec2fd54a19e278eb650f2845cea85afc3d17f3d7f94" exitCode=0
Sep 15 12:34:38.005584 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:38.005455    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c" event={"ID":"096c778f-6bb8-45fc-b3eb-05bc59efdafc","Type":"ContainerDied","Data":"83320d89dcce932a75c72ec2fd54a19e278eb650f2845cea85afc3d17f3d7f94"}
Sep 15 12:34:38.438986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:38.438963    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:38.545327 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:38.545230    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/096c778f-6bb8-45fc-b3eb-05bc59efdafc-config-file\") pod \"096c778f-6bb8-45fc-b3eb-05bc59efdafc\" (UID: \"096c778f-6bb8-45fc-b3eb-05bc59efdafc\") "
Sep 15 12:34:38.545694 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:38.545380    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8xl4\" (UniqueName: \"kubernetes.io/projected/096c778f-6bb8-45fc-b3eb-05bc59efdafc-kube-api-access-n8xl4\") pod \"096c778f-6bb8-45fc-b3eb-05bc59efdafc\" (UID: \"096c778f-6bb8-45fc-b3eb-05bc59efdafc\") "
Sep 15 12:34:38.545694 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:38.545467    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/096c778f-6bb8-45fc-b3eb-05bc59efdafc-config-file" (OuterVolumeSpecName: "config-file") pod "096c778f-6bb8-45fc-b3eb-05bc59efdafc" (UID: "096c778f-6bb8-45fc-b3eb-05bc59efdafc"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:34:38.545694 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:38.545618    2586 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/096c778f-6bb8-45fc-b3eb-05bc59efdafc-config-file\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:34:38.547392 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:38.547363    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/096c778f-6bb8-45fc-b3eb-05bc59efdafc-kube-api-access-n8xl4" (OuterVolumeSpecName: "kube-api-access-n8xl4") pod "096c778f-6bb8-45fc-b3eb-05bc59efdafc" (UID: "096c778f-6bb8-45fc-b3eb-05bc59efdafc"). InnerVolumeSpecName "kube-api-access-n8xl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:34:38.646226 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:38.646147    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8xl4\" (UniqueName: \"kubernetes.io/projected/096c778f-6bb8-45fc-b3eb-05bc59efdafc-kube-api-access-n8xl4\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:34:39.011342 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:39.011312    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c"
Sep 15 12:34:39.011534 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:39.011341    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4bh6c" event={"ID":"096c778f-6bb8-45fc-b3eb-05bc59efdafc","Type":"ContainerDied","Data":"8f5aa6d7d8d38daefa66531d295db1922b5849fc394d7beb5004a63b680102ad"}
Sep 15 12:34:39.011534 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:39.011395    2586 scope.go:117] "RemoveContainer" containerID="83320d89dcce932a75c72ec2fd54a19e278eb650f2845cea85afc3d17f3d7f94"
Sep 15 12:34:39.029920 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:39.029895    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4bh6c"]
Sep 15 12:34:39.035187 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:39.035165    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4bh6c"]
Sep 15 12:34:40.652481 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:40.652450    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="096c778f-6bb8-45fc-b3eb-05bc59efdafc" path="/var/lib/kubelet/pods/096c778f-6bb8-45fc-b3eb-05bc59efdafc/volumes"
Sep 15 12:34:42.920151 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:42.920120    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-f992w"]
Sep 15 12:34:42.920608 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:42.920497    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="096c778f-6bb8-45fc-b3eb-05bc59efdafc" containerName="limitador"
Sep 15 12:34:42.920608 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:42.920509    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="096c778f-6bb8-45fc-b3eb-05bc59efdafc" containerName="limitador"
Sep 15 12:34:42.920608 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:42.920579    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="096c778f-6bb8-45fc-b3eb-05bc59efdafc" containerName="limitador"
Sep 15 12:34:42.924901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:42.924872    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w"
Sep 15 12:34:42.927438 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:42.927415    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-v4rcm\""
Sep 15 12:34:42.927438 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:42.927424    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 15 12:34:42.928381 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:42.928359    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 12:34:42.928479 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:42.928396    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 12:34:42.932823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:42.932554    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-f992w"]
Sep 15 12:34:43.088886 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:43.088854    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2fcce475-67e1-43b9-90d0-356c2d97e8dd-data\") pod \"postgres-69fb4594cf-f992w\" (UID: \"2fcce475-67e1-43b9-90d0-356c2d97e8dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w"
Sep 15 12:34:43.089052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:43.088960    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5nlq\" (UniqueName: \"kubernetes.io/projected/2fcce475-67e1-43b9-90d0-356c2d97e8dd-kube-api-access-x5nlq\") pod \"postgres-69fb4594cf-f992w\" (UID: \"2fcce475-67e1-43b9-90d0-356c2d97e8dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w"
Sep 15 12:34:43.189928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:43.189847    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5nlq\" (UniqueName: \"kubernetes.io/projected/2fcce475-67e1-43b9-90d0-356c2d97e8dd-kube-api-access-x5nlq\") pod \"postgres-69fb4594cf-f992w\" (UID: \"2fcce475-67e1-43b9-90d0-356c2d97e8dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w"
Sep 15 12:34:43.189928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:43.189906    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2fcce475-67e1-43b9-90d0-356c2d97e8dd-data\") pod \"postgres-69fb4594cf-f992w\" (UID: \"2fcce475-67e1-43b9-90d0-356c2d97e8dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w"
Sep 15 12:34:43.190255 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:43.190239    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2fcce475-67e1-43b9-90d0-356c2d97e8dd-data\") pod \"postgres-69fb4594cf-f992w\" (UID: \"2fcce475-67e1-43b9-90d0-356c2d97e8dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w"
Sep 15 12:34:43.197571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:43.197543    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5nlq\" (UniqueName: \"kubernetes.io/projected/2fcce475-67e1-43b9-90d0-356c2d97e8dd-kube-api-access-x5nlq\") pod \"postgres-69fb4594cf-f992w\" (UID: \"2fcce475-67e1-43b9-90d0-356c2d97e8dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w"
Sep 15 12:34:43.236572 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:43.236538    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w"
Sep 15 12:34:43.507090 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:43.507060    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-f992w"]
Sep 15 12:34:43.509643 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:34:43.509618    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fcce475_67e1_43b9_90d0_356c2d97e8dd.slice/crio-238338128d4392695d9ab189c47898c70ae37949958a43adc781783a5b7e0358 WatchSource:0}: Error finding container 238338128d4392695d9ab189c47898c70ae37949958a43adc781783a5b7e0358: Status 404 returned error can't find the container with id 238338128d4392695d9ab189c47898c70ae37949958a43adc781783a5b7e0358
Sep 15 12:34:44.035816 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:44.035768    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w" event={"ID":"2fcce475-67e1-43b9-90d0-356c2d97e8dd","Type":"ContainerStarted","Data":"238338128d4392695d9ab189c47898c70ae37949958a43adc781783a5b7e0358"}
Sep 15 12:34:49.065939 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:49.065901    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w" event={"ID":"2fcce475-67e1-43b9-90d0-356c2d97e8dd","Type":"ContainerStarted","Data":"fc944cbfbd7feaeaa0e1001261f9a84a3429485a2c3dab8ab72149ba5fab187e"}
Sep 15 12:34:49.066383 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:49.066360    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w"
Sep 15 12:34:49.087040 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:49.086983    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w" podStartSLOduration=2.223716752 podStartE2EDuration="7.086969602s" totalImagesPullingTime="4.86325285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:34:49.085264244 +0000 UTC m=+556.897175686" watchObservedRunningTime="2026-09-15 12:34:49.086969602 +0000 UTC m=+556.898881043"
Sep 15 12:34:55.100229 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:55.100198    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f992w"
Sep 15 12:34:55.935989 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:55.935956    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6db9f6b58f-qp92r"]
Sep 15 12:34:55.941411 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:55.941388    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db9f6b58f-qp92r"
Sep 15 12:34:55.943758 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:55.943733    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 15 12:34:55.945445 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:55.945421    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6db9f6b58f-qp92r"]
Sep 15 12:34:56.007070 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:56.007033    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjkrm\" (UniqueName: \"kubernetes.io/projected/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-kube-api-access-wjkrm\") pod \"authorino-6db9f6b58f-qp92r\" (UID: \"cbf38bc2-c292-41d4-b2fa-60e4f28286d8\") " pod="kuadrant-system/authorino-6db9f6b58f-qp92r"
Sep 15 12:34:56.007245 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:56.007098    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-tls-cert\") pod \"authorino-6db9f6b58f-qp92r\" (UID: \"cbf38bc2-c292-41d4-b2fa-60e4f28286d8\") " pod="kuadrant-system/authorino-6db9f6b58f-qp92r"
Sep 15 12:34:56.108304 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:56.108274    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-tls-cert\") pod \"authorino-6db9f6b58f-qp92r\" (UID: \"cbf38bc2-c292-41d4-b2fa-60e4f28286d8\") " pod="kuadrant-system/authorino-6db9f6b58f-qp92r"
Sep 15 12:34:56.108690 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:56.108369    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjkrm\" (UniqueName: \"kubernetes.io/projected/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-kube-api-access-wjkrm\") pod \"authorino-6db9f6b58f-qp92r\" (UID: \"cbf38bc2-c292-41d4-b2fa-60e4f28286d8\") " pod="kuadrant-system/authorino-6db9f6b58f-qp92r"
Sep 15 12:34:56.110955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:56.110931    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-tls-cert\") pod \"authorino-6db9f6b58f-qp92r\" (UID: \"cbf38bc2-c292-41d4-b2fa-60e4f28286d8\") " pod="kuadrant-system/authorino-6db9f6b58f-qp92r"
Sep 15 12:34:56.116431 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:56.116402    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjkrm\" (UniqueName: \"kubernetes.io/projected/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-kube-api-access-wjkrm\") pod \"authorino-6db9f6b58f-qp92r\" (UID: \"cbf38bc2-c292-41d4-b2fa-60e4f28286d8\") " pod="kuadrant-system/authorino-6db9f6b58f-qp92r"
Sep 15 12:34:56.251735 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:56.251704    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db9f6b58f-qp92r"
Sep 15 12:34:56.520757 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:56.520728    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6db9f6b58f-qp92r"]
Sep 15 12:34:56.522793 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:34:56.522751    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbf38bc2_c292_41d4_b2fa_60e4f28286d8.slice/crio-28e5aee28e9f1125df45f70d156a633ec37510b3c8837d7ce35272699b7a47af WatchSource:0}: Error finding container 28e5aee28e9f1125df45f70d156a633ec37510b3c8837d7ce35272699b7a47af: Status 404 returned error can't find the container with id 28e5aee28e9f1125df45f70d156a633ec37510b3c8837d7ce35272699b7a47af
Sep 15 12:34:57.108334 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:57.108295    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6db9f6b58f-qp92r" event={"ID":"cbf38bc2-c292-41d4-b2fa-60e4f28286d8","Type":"ContainerStarted","Data":"176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8"}
Sep 15 12:34:57.108334 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:57.108340    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6db9f6b58f-qp92r" event={"ID":"cbf38bc2-c292-41d4-b2fa-60e4f28286d8","Type":"ContainerStarted","Data":"28e5aee28e9f1125df45f70d156a633ec37510b3c8837d7ce35272699b7a47af"}
Sep 15 12:34:57.122984 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:57.122930    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6db9f6b58f-qp92r" podStartSLOduration=1.627362562 podStartE2EDuration="2.122910568s" totalImagesPullingTime="495.548006ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:34:57.122254888 +0000 UTC m=+564.934166331" watchObservedRunningTime="2026-09-15 12:34:57.122910568 +0000 UTC m=+564.934822009"
Sep 15 12:34:57.149527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:57.149092    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-l9bj5"]
Sep 15 12:34:57.149527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:57.149354    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-l9bj5" podUID="4ed1a6e4-1188-4a88-a3e4-00b31892fce3" containerName="authorino" containerID="cri-o://1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec" gracePeriod=30
Sep 15 12:34:57.394319 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:57.394295    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-l9bj5"
Sep 15 12:34:57.418610 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:57.418577    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k467b\" (UniqueName: \"kubernetes.io/projected/4ed1a6e4-1188-4a88-a3e4-00b31892fce3-kube-api-access-k467b\") pod \"4ed1a6e4-1188-4a88-a3e4-00b31892fce3\" (UID: \"4ed1a6e4-1188-4a88-a3e4-00b31892fce3\") "
Sep 15 12:34:57.420579 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:57.420553    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ed1a6e4-1188-4a88-a3e4-00b31892fce3-kube-api-access-k467b" (OuterVolumeSpecName: "kube-api-access-k467b") pod "4ed1a6e4-1188-4a88-a3e4-00b31892fce3" (UID: "4ed1a6e4-1188-4a88-a3e4-00b31892fce3"). InnerVolumeSpecName "kube-api-access-k467b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:34:57.519869 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:57.519831    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k467b\" (UniqueName: \"kubernetes.io/projected/4ed1a6e4-1188-4a88-a3e4-00b31892fce3-kube-api-access-k467b\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:34:58.114352 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:58.114320    2586 generic.go:356] "Generic (PLEG): container finished" podID="4ed1a6e4-1188-4a88-a3e4-00b31892fce3" containerID="1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec" exitCode=0
Sep 15 12:34:58.114823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:58.114365    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-l9bj5"
Sep 15 12:34:58.114823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:58.114409    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-l9bj5" event={"ID":"4ed1a6e4-1188-4a88-a3e4-00b31892fce3","Type":"ContainerDied","Data":"1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec"}
Sep 15 12:34:58.114823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:58.114454    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-l9bj5" event={"ID":"4ed1a6e4-1188-4a88-a3e4-00b31892fce3","Type":"ContainerDied","Data":"467e77f1b6beb4cf8f4362f724f7cfbee4182a9993a7978ecbdea325ddbcd333"}
Sep 15 12:34:58.114823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:58.114471    2586 scope.go:117] "RemoveContainer" containerID="1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec"
Sep 15 12:34:58.123088 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:58.123069    2586 scope.go:117] "RemoveContainer" containerID="1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec"
Sep 15 12:34:58.123357 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:34:58.123339    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec\": container with ID starting with 1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec not found: ID does not exist" containerID="1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec"
Sep 15 12:34:58.123404 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:58.123366    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec"} err="failed to get container status \"1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec\": rpc error: code = NotFound desc = could not find container \"1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec\": container with ID starting with 1636dd4b20dabc5700c4336347c0c493627ce91937a51023f617b57be75840ec not found: ID does not exist"
Sep 15 12:34:58.134093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:58.134069    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-l9bj5"]
Sep 15 12:34:58.139874 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:58.139852    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-l9bj5"]
Sep 15 12:34:58.652534 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:34:58.652505    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ed1a6e4-1188-4a88-a3e4-00b31892fce3" path="/var/lib/kubelet/pods/4ed1a6e4-1188-4a88-a3e4-00b31892fce3/volumes"
Sep 15 12:35:02.019636 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.019604    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-88b49f7cd-6ctdj"]
Sep 15 12:35:02.020170 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.020151    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed1a6e4-1188-4a88-a3e4-00b31892fce3" containerName="authorino"
Sep 15 12:35:02.020250 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.020174    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed1a6e4-1188-4a88-a3e4-00b31892fce3" containerName="authorino"
Sep 15 12:35:02.020303 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.020291    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ed1a6e4-1188-4a88-a3e4-00b31892fce3" containerName="authorino"
Sep 15 12:35:02.023173 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.023150    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.025734 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.025710    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 12:35:02.025907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.025750    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 12:35:02.025907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.025886    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m2skh\""
Sep 15 12:35:02.026621 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.026605    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 12:35:02.035123 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.035091    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-88b49f7cd-6ctdj"]
Sep 15 12:35:02.060075 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.060049    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-webhook-cert\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.060208 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.060081    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4r2l\" (UniqueName: \"kubernetes.io/projected/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-kube-api-access-n4r2l\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.060208 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.060176    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-metrics-certs\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.160902 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.160864    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-metrics-certs\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.161110 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.160945    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-webhook-cert\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.161110 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.160970    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4r2l\" (UniqueName: \"kubernetes.io/projected/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-kube-api-access-n4r2l\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.161110 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:02.161015    2586 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 15 12:35:02.161110 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:02.161077    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-metrics-certs podName:cfc76a52-54ac-4fa0-a610-dd5ecffefe9e nodeName:}" failed. No retries permitted until 2026-09-15 12:35:02.661062366 +0000 UTC m=+570.472973790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-metrics-certs") pod "maas-controller-88b49f7cd-6ctdj" (UID: "cfc76a52-54ac-4fa0-a610-dd5ecffefe9e") : secret "maas-controller-metrics-tls" not found
Sep 15 12:35:02.161110 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:02.161083    2586 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 15 12:35:02.161329 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:02.161137    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-webhook-cert podName:cfc76a52-54ac-4fa0-a610-dd5ecffefe9e nodeName:}" failed. No retries permitted until 2026-09-15 12:35:02.661119063 +0000 UTC m=+570.473030497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-webhook-cert") pod "maas-controller-88b49f7cd-6ctdj" (UID: "cfc76a52-54ac-4fa0-a610-dd5ecffefe9e") : secret "maas-controller-webhook-cert" not found
Sep 15 12:35:02.170257 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.170222    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4r2l\" (UniqueName: \"kubernetes.io/projected/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-kube-api-access-n4r2l\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.191745 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.191719    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65f48575b5-qhgwt"]
Sep 15 12:35:02.194189 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.194172    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.208517 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.208488    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65f48575b5-qhgwt"]
Sep 15 12:35:02.262035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.261997    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-webhook-cert\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.262196 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.262060    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc5bf\" (UniqueName: \"kubernetes.io/projected/99d65925-7457-4832-b0e5-8cb75689824f-kube-api-access-dc5bf\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.262196 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.262111    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-metrics-certs\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.362703 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.362620    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-webhook-cert\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.362703 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.362677    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc5bf\" (UniqueName: \"kubernetes.io/projected/99d65925-7457-4832-b0e5-8cb75689824f-kube-api-access-dc5bf\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.362975 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.362715    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-metrics-certs\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.362975 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:02.362809    2586 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 15 12:35:02.362975 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:02.362872    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-webhook-cert podName:99d65925-7457-4832-b0e5-8cb75689824f nodeName:}" failed. No retries permitted until 2026-09-15 12:35:02.862855033 +0000 UTC m=+570.674766457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-webhook-cert") pod "maas-controller-65f48575b5-qhgwt" (UID: "99d65925-7457-4832-b0e5-8cb75689824f") : secret "maas-controller-webhook-cert" not found
Sep 15 12:35:02.362975 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:02.362875    2586 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 15 12:35:02.362975 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:02.362933    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-metrics-certs podName:99d65925-7457-4832-b0e5-8cb75689824f nodeName:}" failed. No retries permitted until 2026-09-15 12:35:02.862916387 +0000 UTC m=+570.674827823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-metrics-certs") pod "maas-controller-65f48575b5-qhgwt" (UID: "99d65925-7457-4832-b0e5-8cb75689824f") : secret "maas-controller-metrics-tls" not found
Sep 15 12:35:02.373603 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.373577    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc5bf\" (UniqueName: \"kubernetes.io/projected/99d65925-7457-4832-b0e5-8cb75689824f-kube-api-access-dc5bf\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.666381 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.666282    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-metrics-certs\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.666558 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.666390    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-webhook-cert\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.668845 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.668821    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-metrics-certs\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.669009 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.668989    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-webhook-cert\") pod \"maas-controller-88b49f7cd-6ctdj\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") " pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:02.868630 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.868584    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-webhook-cert\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.868869 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.868683    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-metrics-certs\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.871111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.871082    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-metrics-certs\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.871227 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.871164    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-webhook-cert\") pod \"maas-controller-65f48575b5-qhgwt\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") " pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:02.934256 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:02.934179    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:03.105545 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:03.105510    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:03.203219 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:03.203193    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-88b49f7cd-6ctdj"]
Sep 15 12:35:03.205380 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:35:03.205347    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfc76a52_54ac_4fa0_a610_dd5ecffefe9e.slice/crio-4f96f78e931a0c3e08372a8e96d35a26bd287675b1ef5a7c2211c078840d3897 WatchSource:0}: Error finding container 4f96f78e931a0c3e08372a8e96d35a26bd287675b1ef5a7c2211c078840d3897: Status 404 returned error can't find the container with id 4f96f78e931a0c3e08372a8e96d35a26bd287675b1ef5a7c2211c078840d3897
Sep 15 12:35:03.375332 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:03.375302    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65f48575b5-qhgwt"]
Sep 15 12:35:03.376504 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:35:03.376475    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d65925_7457_4832_b0e5_8cb75689824f.slice/crio-e1aac0e2f19f9a3a1f7f86a5d063b13f0c0e41d95a92c631fa6f95a9a587118e WatchSource:0}: Error finding container e1aac0e2f19f9a3a1f7f86a5d063b13f0c0e41d95a92c631fa6f95a9a587118e: Status 404 returned error can't find the container with id e1aac0e2f19f9a3a1f7f86a5d063b13f0c0e41d95a92c631fa6f95a9a587118e
Sep 15 12:35:04.174055 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:04.174013    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88b49f7cd-6ctdj" event={"ID":"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e","Type":"ContainerStarted","Data":"4f96f78e931a0c3e08372a8e96d35a26bd287675b1ef5a7c2211c078840d3897"}
Sep 15 12:35:04.175734 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:04.175701    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65f48575b5-qhgwt" event={"ID":"99d65925-7457-4832-b0e5-8cb75689824f","Type":"ContainerStarted","Data":"e1aac0e2f19f9a3a1f7f86a5d063b13f0c0e41d95a92c631fa6f95a9a587118e"}
Sep 15 12:35:06.186969 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:06.186868    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88b49f7cd-6ctdj" event={"ID":"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e","Type":"ContainerStarted","Data":"129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08"}
Sep 15 12:35:06.186969 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:06.186940    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:06.188301 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:06.188274    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65f48575b5-qhgwt" event={"ID":"99d65925-7457-4832-b0e5-8cb75689824f","Type":"ContainerStarted","Data":"9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1"}
Sep 15 12:35:06.188413 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:06.188403    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:06.209366 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:06.209317    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-88b49f7cd-6ctdj" podStartSLOduration=1.530958952 podStartE2EDuration="4.209299888s" totalImagesPullingTime="2.678340936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:35:06.207419017 +0000 UTC m=+574.019330461" watchObservedRunningTime="2026-09-15 12:35:06.209299888 +0000 UTC m=+574.021211330"
Sep 15 12:35:06.224254 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:06.224212    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65f48575b5-qhgwt" podStartSLOduration=1.722240265 podStartE2EDuration="4.224196633s" totalImagesPullingTime="2.501956368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:35:06.223389377 +0000 UTC m=+574.035300856" watchObservedRunningTime="2026-09-15 12:35:06.224196633 +0000 UTC m=+574.036108074"
Sep 15 12:35:17.194848 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.194820    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:17.195358 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.194975    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 12:35:17.256588 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.256554    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-88b49f7cd-6ctdj"]
Sep 15 12:35:17.256810 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.256760    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-88b49f7cd-6ctdj" podUID="cfc76a52-54ac-4fa0-a610-dd5ecffefe9e" containerName="manager" containerID="cri-o://129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08" gracePeriod=30
Sep 15 12:35:17.503226 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.503203    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:17.598466 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.598437    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4r2l\" (UniqueName: \"kubernetes.io/projected/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-kube-api-access-n4r2l\") pod \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") "
Sep 15 12:35:17.598606 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.598486    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-webhook-cert\") pod \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") "
Sep 15 12:35:17.598606 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.598514    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-metrics-certs\") pod \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\" (UID: \"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e\") "
Sep 15 12:35:17.600697 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.600667    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cfc76a52-54ac-4fa0-a610-dd5ecffefe9e" (UID: "cfc76a52-54ac-4fa0-a610-dd5ecffefe9e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:35:17.600827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.600697    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cfc76a52-54ac-4fa0-a610-dd5ecffefe9e" (UID: "cfc76a52-54ac-4fa0-a610-dd5ecffefe9e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:35:17.600827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.600730    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-kube-api-access-n4r2l" (OuterVolumeSpecName: "kube-api-access-n4r2l") pod "cfc76a52-54ac-4fa0-a610-dd5ecffefe9e" (UID: "cfc76a52-54ac-4fa0-a610-dd5ecffefe9e"). InnerVolumeSpecName "kube-api-access-n4r2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:35:17.700209 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.700181    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4r2l\" (UniqueName: \"kubernetes.io/projected/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-kube-api-access-n4r2l\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:35:17.700209 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.700204    2586 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-webhook-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:35:17.700209 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:17.700213    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:35:18.246354 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:18.246320    2586 generic.go:356] "Generic (PLEG): container finished" podID="cfc76a52-54ac-4fa0-a610-dd5ecffefe9e" containerID="129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08" exitCode=0
Sep 15 12:35:18.246749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:18.246373    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88b49f7cd-6ctdj"
Sep 15 12:35:18.246749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:18.246405    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88b49f7cd-6ctdj" event={"ID":"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e","Type":"ContainerDied","Data":"129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08"}
Sep 15 12:35:18.246749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:18.246491    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88b49f7cd-6ctdj" event={"ID":"cfc76a52-54ac-4fa0-a610-dd5ecffefe9e","Type":"ContainerDied","Data":"4f96f78e931a0c3e08372a8e96d35a26bd287675b1ef5a7c2211c078840d3897"}
Sep 15 12:35:18.246749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:18.246520    2586 scope.go:117] "RemoveContainer" containerID="129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08"
Sep 15 12:35:18.255582 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:18.255566    2586 scope.go:117] "RemoveContainer" containerID="129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08"
Sep 15 12:35:18.255861 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:18.255836    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08\": container with ID starting with 129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08 not found: ID does not exist" containerID="129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08"
Sep 15 12:35:18.255954 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:18.255869    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08"} err="failed to get container status \"129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08\": rpc error: code = NotFound desc = could not find container \"129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08\": container with ID starting with 129db55bd89a42f7591ff1152e4c05f91e02cb077aee0243953a5b20d8a17b08 not found: ID does not exist"
Sep 15 12:35:18.267150 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:18.267128    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-88b49f7cd-6ctdj"]
Sep 15 12:35:18.270395 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:18.270374    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-88b49f7cd-6ctdj"]
Sep 15 12:35:18.653092 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:18.653018    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfc76a52-54ac-4fa0-a610-dd5ecffefe9e" path="/var/lib/kubelet/pods/cfc76a52-54ac-4fa0-a610-dd5ecffefe9e/volumes"
Sep 15 12:35:39.331343 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.331311    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"]
Sep 15 12:35:39.331820 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.331689    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfc76a52-54ac-4fa0-a610-dd5ecffefe9e" containerName="manager"
Sep 15 12:35:39.331820 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.331700    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc76a52-54ac-4fa0-a610-dd5ecffefe9e" containerName="manager"
Sep 15 12:35:39.331820 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.331757    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfc76a52-54ac-4fa0-a610-dd5ecffefe9e" containerName="manager"
Sep 15 12:35:39.336212 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.336191    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.338806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.338738    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 15 12:35:39.339598 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.339575    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 15 12:35:39.339598 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.339579    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kqdbn\""
Sep 15 12:35:39.345935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.345911    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"]
Sep 15 12:35:39.380228 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.380193    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-openshift-service-ca\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.380228 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.380230    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-maas-api-tls\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.380468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.380296    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-metrics-certs\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.380468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.380335    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcvrj\" (UniqueName: \"kubernetes.io/projected/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-kube-api-access-jcvrj\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.481286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.481250    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-metrics-certs\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.481470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.481322    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcvrj\" (UniqueName: \"kubernetes.io/projected/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-kube-api-access-jcvrj\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.481470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.481364    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-openshift-service-ca\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.481470 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:39.481412    2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 15 12:35:39.481617 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:35:39.481482    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-metrics-certs podName:f5c8cdac-9b87-406f-b77a-cf2a8240c67d nodeName:}" failed. No retries permitted until 2026-09-15 12:35:39.981459868 +0000 UTC m=+607.793371292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-metrics-certs") pod "maas-api-556658ccdd-q62p8" (UID: "f5c8cdac-9b87-406f-b77a-cf2a8240c67d") : secret "maas-api-metrics-tls" not found
Sep 15 12:35:39.481617 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.481500    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-maas-api-tls\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.482115 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.482090    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-openshift-service-ca\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.483976 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.483961    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-maas-api-tls\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.489957 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.489936    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcvrj\" (UniqueName: \"kubernetes.io/projected/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-kube-api-access-jcvrj\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.883175 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.883137    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"]
Sep 15 12:35:39.885543 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.885525    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"
Sep 15 12:35:39.888056 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.888031    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 15 12:35:39.888174 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.888148    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fr74p\""
Sep 15 12:35:39.898270 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.898239    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"]
Sep 15 12:35:39.987100 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.987061    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pwtj\" (UniqueName: \"kubernetes.io/projected/1a21963b-0a26-4775-83a1-ccbcfbc6a38b-kube-api-access-4pwtj\") pod \"payload-pre-processing-59fbc47bf8-k7cfc\" (UID: \"1a21963b-0a26-4775-83a1-ccbcfbc6a38b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"
Sep 15 12:35:39.987273 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.987118    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a21963b-0a26-4775-83a1-ccbcfbc6a38b-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-k7cfc\" (UID: \"1a21963b-0a26-4775-83a1-ccbcfbc6a38b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"
Sep 15 12:35:39.987273 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.987248    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-metrics-certs\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:39.989735 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:39.989710    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-metrics-certs\") pod \"maas-api-556658ccdd-q62p8\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") " pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:40.088619 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:40.088575    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pwtj\" (UniqueName: \"kubernetes.io/projected/1a21963b-0a26-4775-83a1-ccbcfbc6a38b-kube-api-access-4pwtj\") pod \"payload-pre-processing-59fbc47bf8-k7cfc\" (UID: \"1a21963b-0a26-4775-83a1-ccbcfbc6a38b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"
Sep 15 12:35:40.088765 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:40.088631    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a21963b-0a26-4775-83a1-ccbcfbc6a38b-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-k7cfc\" (UID: \"1a21963b-0a26-4775-83a1-ccbcfbc6a38b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"
Sep 15 12:35:40.089348 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:40.089327    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a21963b-0a26-4775-83a1-ccbcfbc6a38b-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-k7cfc\" (UID: \"1a21963b-0a26-4775-83a1-ccbcfbc6a38b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"
Sep 15 12:35:40.097724 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:40.097694    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pwtj\" (UniqueName: \"kubernetes.io/projected/1a21963b-0a26-4775-83a1-ccbcfbc6a38b-kube-api-access-4pwtj\") pod \"payload-pre-processing-59fbc47bf8-k7cfc\" (UID: \"1a21963b-0a26-4775-83a1-ccbcfbc6a38b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"
Sep 15 12:35:40.198755 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:40.198656    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"
Sep 15 12:35:40.248653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:40.248613    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:40.483525 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:40.483500    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"]
Sep 15 12:35:40.485982 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:35:40.485952    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a21963b_0a26_4775_83a1_ccbcfbc6a38b.slice/crio-6605d7d2f6decafdb8d42a987b1ae5355acaad53fec43a9ac559ae2e0eb83a69 WatchSource:0}: Error finding container 6605d7d2f6decafdb8d42a987b1ae5355acaad53fec43a9ac559ae2e0eb83a69: Status 404 returned error can't find the container with id 6605d7d2f6decafdb8d42a987b1ae5355acaad53fec43a9ac559ae2e0eb83a69
Sep 15 12:35:40.543818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:40.539354    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"]
Sep 15 12:35:41.365661 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:41.365618    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc" event={"ID":"1a21963b-0a26-4775-83a1-ccbcfbc6a38b","Type":"ContainerStarted","Data":"6605d7d2f6decafdb8d42a987b1ae5355acaad53fec43a9ac559ae2e0eb83a69"}
Sep 15 12:35:41.367477 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:41.367447    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8" event={"ID":"f5c8cdac-9b87-406f-b77a-cf2a8240c67d","Type":"ContainerStarted","Data":"6ed062d87a699bf8e173443151040d25553d6d1d70561aed228f747951d374cb"}
Sep 15 12:35:44.385557 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:44.385519    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8" event={"ID":"f5c8cdac-9b87-406f-b77a-cf2a8240c67d","Type":"ContainerStarted","Data":"2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275"}
Sep 15 12:35:44.386023 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:44.385592    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:44.387180 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:44.387156    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc" event={"ID":"1a21963b-0a26-4775-83a1-ccbcfbc6a38b","Type":"ContainerStarted","Data":"4f965c5dd69267e1ff3339463566b457f97be6b38cc1c8a820c44403e8160b70"}
Sep 15 12:35:44.387280 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:44.387207    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"
Sep 15 12:35:44.404803 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:44.403722    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8" podStartSLOduration=2.1820496560000002 podStartE2EDuration="5.403703376s" totalImagesPullingTime="3.22165372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:35:44.403146863 +0000 UTC m=+612.215058307" watchObservedRunningTime="2026-09-15 12:35:44.403703376 +0000 UTC m=+612.215614819"
Sep 15 12:35:49.414721 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:49.414692    2586 generic.go:356] "Generic (PLEG): container finished" podID="d26fc663-72b4-4ff5-9e56-3d5eb6603e29" containerID="602968c88322e564710f7c09d0befc224a1fca9e44e3f7d9ced149f5f6353685" exitCode=0
Sep 15 12:35:49.415138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:49.414762    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl" event={"ID":"d26fc663-72b4-4ff5-9e56-3d5eb6603e29","Type":"ContainerDied","Data":"602968c88322e564710f7c09d0befc224a1fca9e44e3f7d9ced149f5f6353685"}
Sep 15 12:35:49.415199 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:49.415182    2586 scope.go:117] "RemoveContainer" containerID="602968c88322e564710f7c09d0befc224a1fca9e44e3f7d9ced149f5f6353685"
Sep 15 12:35:49.416194 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:49.416165    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:35:49.416255 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:49.416236    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:35:49.416289 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:49.416268    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:35:49.437246 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:49.437206    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc" podStartSLOduration=7.086449573 podStartE2EDuration="10.437191248s" totalImagesPullingTime="3.350741675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:35:44.423656089 +0000 UTC m=+612.235567549" watchObservedRunningTime="2026-09-15 12:35:49.437191248 +0000 UTC m=+617.249102747"
Sep 15 12:35:50.405766 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:50.405067    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 12:35:50.423918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:50.423880    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl" event={"ID":"d26fc663-72b4-4ff5-9e56-3d5eb6603e29","Type":"ContainerStarted","Data":"6d8830015dfc9b410806abd3ef42a516ab5686e5681f7cab4f3978eecd4d7e17"}
Sep 15 12:35:51.385690 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:51.385654    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:35:51.386611 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:51.386574    2586 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-dtgvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 15 12:35:51.386731 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:51.386640    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl" podUID="d26fc663-72b4-4ff5-9e56-3d5eb6603e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 15 12:35:52.385536 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:52.385492    2586 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-dtgvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 15 12:35:52.385958 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:52.385551    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl" podUID="d26fc663-72b4-4ff5-9e56-3d5eb6603e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 15 12:35:53.385130 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:53.385094    2586 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-dtgvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 15 12:35:53.385304 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:53.385159    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl" podUID="d26fc663-72b4-4ff5-9e56-3d5eb6603e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 15 12:35:54.385290 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:54.385250    2586 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-dtgvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 15 12:35:54.385731 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:54.385309    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl" podUID="d26fc663-72b4-4ff5-9e56-3d5eb6603e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 15 12:35:55.385299 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:55.385224    2586 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-dtgvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 15 12:35:55.385299 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:55.385279    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl" podUID="d26fc663-72b4-4ff5-9e56-3d5eb6603e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 15 12:35:55.394307 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:55.394287    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k7cfc"
Sep 15 12:35:55.867063 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:55.867029    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:35:56.388941 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:56.388911    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:35:56.452326 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:35:56.452302    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-dtgvl"
Sep 15 12:36:05.550507 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.550477    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"]
Sep 15 12:36:05.553177 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.553157    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.555519 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.555498    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 15 12:36:05.555635 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.555527    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 15 12:36:05.556266 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.556247    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 15 12:36:05.556354 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.556269    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2598m\""
Sep 15 12:36:05.567545 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.567522    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"]
Sep 15 12:36:05.642174 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.642144    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.642307 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.642184    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d304d1ba-1999-4766-8df9-b0ad2d9827e9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.642307 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.642212    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.642384 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.642306    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.642384 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.642348    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.642384 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.642374    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqjbv\" (UniqueName: \"kubernetes.io/projected/d304d1ba-1999-4766-8df9-b0ad2d9827e9-kube-api-access-jqjbv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.743633 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.743597    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqjbv\" (UniqueName: \"kubernetes.io/projected/d304d1ba-1999-4766-8df9-b0ad2d9827e9-kube-api-access-jqjbv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.743820 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.743639    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.743820 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.743669    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d304d1ba-1999-4766-8df9-b0ad2d9827e9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.743820 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.743697    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.743820 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.743738    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.744022 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.743912    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.744096 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.744077    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.744151 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.744104    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.744269 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.744250    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.746029 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.746009    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d304d1ba-1999-4766-8df9-b0ad2d9827e9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.746264 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.746248    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d304d1ba-1999-4766-8df9-b0ad2d9827e9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.751080 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.751063    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqjbv\" (UniqueName: \"kubernetes.io/projected/d304d1ba-1999-4766-8df9-b0ad2d9827e9-kube-api-access-jqjbv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qm4x6\" (UID: \"d304d1ba-1999-4766-8df9-b0ad2d9827e9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:05.863328 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:05.863259    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:06.132093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:06.132052    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"]
Sep 15 12:36:06.134755 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:36:06.134723    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd304d1ba_1999_4766_8df9_b0ad2d9827e9.slice/crio-f2f3620a35f92a5dbdea7bc25005c8b15426739d3b5ec50ebae3eeed2bfcb506 WatchSource:0}: Error finding container f2f3620a35f92a5dbdea7bc25005c8b15426739d3b5ec50ebae3eeed2bfcb506: Status 404 returned error can't find the container with id f2f3620a35f92a5dbdea7bc25005c8b15426739d3b5ec50ebae3eeed2bfcb506
Sep 15 12:36:06.500261 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:06.500225    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6" event={"ID":"d304d1ba-1999-4766-8df9-b0ad2d9827e9","Type":"ContainerStarted","Data":"f2f3620a35f92a5dbdea7bc25005c8b15426739d3b5ec50ebae3eeed2bfcb506"}
Sep 15 12:36:07.506426 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:07.506391    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6" event={"ID":"d304d1ba-1999-4766-8df9-b0ad2d9827e9","Type":"ContainerStarted","Data":"045565fb5cd1b840cc82c945320a5db952fce34b8a81ba4f57848f39c36d46bc"}
Sep 15 12:36:07.522843 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:07.522768    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6" podStartSLOduration=1.317134593 podStartE2EDuration="2.5227516s" totalImagesPullingTime="1.205617007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:36:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:36:07.522241368 +0000 UTC m=+635.334152811" watchObservedRunningTime="2026-09-15 12:36:07.5227516 +0000 UTC m=+635.334663041"
Sep 15 12:36:15.863526 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:15.863485    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:15.863989 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:15.863550    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:15.876018 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:15.875996    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:16.558214 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:16.558183    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qm4x6"
Sep 15 12:36:37.657944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.657910    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"]
Sep 15 12:36:37.660205 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.660189    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.662377 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.662356    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 15 12:36:37.669143 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.669122    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"]
Sep 15 12:36:37.733913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.733884    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.734070 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.733923    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.734070 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.733948    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f440992c-7441-4099-8b1f-1a9f07527ce5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.734070 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.734049    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.734230 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.734084    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.734230 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.734125    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrt6f\" (UniqueName: \"kubernetes.io/projected/f440992c-7441-4099-8b1f-1a9f07527ce5-kube-api-access-qrt6f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.835518 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.835488    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.835518 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.835520    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f440992c-7441-4099-8b1f-1a9f07527ce5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.835747 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.835588    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.835747 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.835611    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.835747 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.835647    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrt6f\" (UniqueName: \"kubernetes.io/projected/f440992c-7441-4099-8b1f-1a9f07527ce5-kube-api-access-qrt6f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.835747 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.835701    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.836064 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.836042    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.836159 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.836069    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.836159 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.836092    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.837741 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.837721    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f440992c-7441-4099-8b1f-1a9f07527ce5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.838108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.838091    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f440992c-7441-4099-8b1f-1a9f07527ce5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.847305 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.847279    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrt6f\" (UniqueName: \"kubernetes.io/projected/f440992c-7441-4099-8b1f-1a9f07527ce5-kube-api-access-qrt6f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hm4t7\" (UID: \"f440992c-7441-4099-8b1f-1a9f07527ce5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:37.971052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:37.970983    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:38.240728 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:38.240703    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"]
Sep 15 12:36:38.243433 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:36:38.243407    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf440992c_7441_4099_8b1f_1a9f07527ce5.slice/crio-91094788248ca46a7ab2c348320cf7c77161f0b8ac43d8d178fbf5416a555183 WatchSource:0}: Error finding container 91094788248ca46a7ab2c348320cf7c77161f0b8ac43d8d178fbf5416a555183: Status 404 returned error can't find the container with id 91094788248ca46a7ab2c348320cf7c77161f0b8ac43d8d178fbf5416a555183
Sep 15 12:36:38.244739 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:38.244722    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 12:36:38.660772 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:38.660733    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7" event={"ID":"f440992c-7441-4099-8b1f-1a9f07527ce5","Type":"ContainerStarted","Data":"da84e1b1156c0d5d15b104ea15c46eb2b4350785e1de0a76c91b1bfa1bc97aeb"}
Sep 15 12:36:38.661144 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:38.660805    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7" event={"ID":"f440992c-7441-4099-8b1f-1a9f07527ce5","Type":"ContainerStarted","Data":"91094788248ca46a7ab2c348320cf7c77161f0b8ac43d8d178fbf5416a555183"}
Sep 15 12:36:38.679737 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:38.679662    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7" podStartSLOduration=1.438777782 podStartE2EDuration="1.679644537s" totalImagesPullingTime="240.866755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:36:38.679204118 +0000 UTC m=+666.491115562" watchObservedRunningTime="2026-09-15 12:36:38.679644537 +0000 UTC m=+666.491555979"
Sep 15 12:36:47.971504 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:47.971467    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:47.972056 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:47.971815    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:47.984030 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:47.984007    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:36:48.716024 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:36:48.715996    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hm4t7"
Sep 15 12:38:03.573429 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:03.573392    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6db9f6b58f-qp92r"]
Sep 15 12:38:03.573989 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:03.573584    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6db9f6b58f-qp92r" podUID="cbf38bc2-c292-41d4-b2fa-60e4f28286d8" containerName="authorino" containerID="cri-o://176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8" gracePeriod=30
Sep 15 12:38:03.826526 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:03.826476    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db9f6b58f-qp92r"
Sep 15 12:38:03.857413 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:03.857387    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-tls-cert\") pod \"cbf38bc2-c292-41d4-b2fa-60e4f28286d8\" (UID: \"cbf38bc2-c292-41d4-b2fa-60e4f28286d8\") "
Sep 15 12:38:03.857548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:03.857453    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjkrm\" (UniqueName: \"kubernetes.io/projected/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-kube-api-access-wjkrm\") pod \"cbf38bc2-c292-41d4-b2fa-60e4f28286d8\" (UID: \"cbf38bc2-c292-41d4-b2fa-60e4f28286d8\") "
Sep 15 12:38:03.859420 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:03.859389    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-kube-api-access-wjkrm" (OuterVolumeSpecName: "kube-api-access-wjkrm") pod "cbf38bc2-c292-41d4-b2fa-60e4f28286d8" (UID: "cbf38bc2-c292-41d4-b2fa-60e4f28286d8"). InnerVolumeSpecName "kube-api-access-wjkrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:38:03.867661 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:03.867630    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "cbf38bc2-c292-41d4-b2fa-60e4f28286d8" (UID: "cbf38bc2-c292-41d4-b2fa-60e4f28286d8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:38:03.958913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:03.958885    2586 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-tls-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:38:03.958913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:03.958909    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjkrm\" (UniqueName: \"kubernetes.io/projected/cbf38bc2-c292-41d4-b2fa-60e4f28286d8-kube-api-access-wjkrm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:38:04.064548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:04.064519    2586 generic.go:356] "Generic (PLEG): container finished" podID="cbf38bc2-c292-41d4-b2fa-60e4f28286d8" containerID="176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8" exitCode=0
Sep 15 12:38:04.064708 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:04.064555    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db9f6b58f-qp92r"
Sep 15 12:38:04.064708 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:04.064587    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6db9f6b58f-qp92r" event={"ID":"cbf38bc2-c292-41d4-b2fa-60e4f28286d8","Type":"ContainerDied","Data":"176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8"}
Sep 15 12:38:04.064708 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:04.064619    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6db9f6b58f-qp92r" event={"ID":"cbf38bc2-c292-41d4-b2fa-60e4f28286d8","Type":"ContainerDied","Data":"28e5aee28e9f1125df45f70d156a633ec37510b3c8837d7ce35272699b7a47af"}
Sep 15 12:38:04.064708 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:04.064637    2586 scope.go:117] "RemoveContainer" containerID="176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8"
Sep 15 12:38:04.073788 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:04.073759    2586 scope.go:117] "RemoveContainer" containerID="176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8"
Sep 15 12:38:04.074087 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:38:04.074070    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8\": container with ID starting with 176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8 not found: ID does not exist" containerID="176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8"
Sep 15 12:38:04.074130 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:04.074097    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8"} err="failed to get container status \"176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8\": rpc error: code = NotFound desc = could not find container \"176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8\": container with ID starting with 176f4a51ac74536c9889aeb4e19d6b1779d5f4f525a797cc888310549a1651e8 not found: ID does not exist"
Sep 15 12:38:04.092851 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:04.092796    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6db9f6b58f-qp92r"]
Sep 15 12:38:04.101274 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:04.101249    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6db9f6b58f-qp92r"]
Sep 15 12:38:04.652682 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:04.652651    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbf38bc2-c292-41d4-b2fa-60e4f28286d8" path="/var/lib/kubelet/pods/cbf38bc2-c292-41d4-b2fa-60e4f28286d8/volumes"
Sep 15 12:38:24.952228 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:24.952196    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"]
Sep 15 12:38:24.952629 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:24.952565    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf38bc2-c292-41d4-b2fa-60e4f28286d8" containerName="authorino"
Sep 15 12:38:24.952629 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:24.952575    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf38bc2-c292-41d4-b2fa-60e4f28286d8" containerName="authorino"
Sep 15 12:38:24.952716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:24.952641    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf38bc2-c292-41d4-b2fa-60e4f28286d8" containerName="authorino"
Sep 15 12:38:24.956933 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:24.956913    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:24.959527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:24.959506    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-ee96ec-openshift-default-dockercfg-lcfxk\""
Sep 15 12:38:24.967773 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:24.967749    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"]
Sep 15 12:38:25.004896 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.004862    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"]
Sep 15 12:38:25.008443 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.008418    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.011050 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.011026    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4609ba-openshift-default-dockercfg-wjmkd\""
Sep 15 12:38:25.021193 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.021163    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"]
Sep 15 12:38:25.039075 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.039048    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-credential-socket\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.039194 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.039085    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-certs\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.039194 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.039110    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-socket\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.039194 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.039136    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-istio-token\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.039318 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.039196    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-envoy\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.039318 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.039222    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-data\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.039642 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.039617    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70eca63a-d3d3-480f-a771-9e59aedee860-istio-podinfo\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.039734 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.039671    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-964fw\" (UniqueName: \"kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-kube-api-access-964fw\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.039804 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.039745    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70eca63a-d3d3-480f-a771-9e59aedee860-istiod-ca-cert\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.140850 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.140817    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70eca63a-d3d3-480f-a771-9e59aedee860-istio-podinfo\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.140850 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.140852    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-964fw\" (UniqueName: \"kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-kube-api-access-964fw\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.140873    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-podinfo\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.141084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.140888    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istiod-ca-cert\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.141084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.140934    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-envoy\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.141084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.140972    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-data\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.141084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.140992    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70eca63a-d3d3-480f-a771-9e59aedee860-istiod-ca-cert\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141015    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-socket\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.141084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141074    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-credential-socket\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.141432 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141100    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-credential-socket\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141432 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141157    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-token\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.141432 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141207    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-certs\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141432 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141242    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-socket\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141432 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141268    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-certs\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.141432 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141294    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckrpf\" (UniqueName: \"kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-kube-api-access-ckrpf\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.141432 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141328    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-istio-token\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141432 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141393    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-envoy\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141432 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141421    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-data\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141458    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-credential-socket\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141554    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-certs\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141616    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-socket\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.141880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141828    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70eca63a-d3d3-480f-a771-9e59aedee860-istiod-ca-cert\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.142032 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.141891    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-data\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.143575 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.143553    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-envoy\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.143964 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.143946    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70eca63a-d3d3-480f-a771-9e59aedee860-istio-podinfo\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.148735 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.148706    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-istio-token\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.148865 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.148801    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-964fw\" (UniqueName: \"kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-kube-api-access-964fw\") pod \"e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") " pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.242416 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242384    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-credential-socket\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.242599 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242428    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-token\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.242599 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242554    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-certs\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.242714 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242603    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckrpf\" (UniqueName: \"kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-kube-api-access-ckrpf\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.242714 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242700    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-podinfo\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.242879 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242734    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istiod-ca-cert\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.242879 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242811    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-envoy\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.242879 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242832    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-credential-socket\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.242879 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242835    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-data\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.243072 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242884    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-certs\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.243072 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.242909    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-socket\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.243175 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.243122    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-data\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.243281 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.243238    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-socket\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.243498 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.243479    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istiod-ca-cert\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.245270 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.245250    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-envoy\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.245484 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.245467    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-podinfo\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.249573 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.249542    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckrpf\" (UniqueName: \"kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-kube-api-access-ckrpf\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.249660 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.249589    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-token\") pod \"e2e-inf-a-4609ba-openshift-default-576898958-qb6w4\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") " pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.268411 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.268386    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:25.321456 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.321427    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:25.543225 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.543201    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"]
Sep 15 12:38:25.545874 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:38:25.545844    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70eca63a_d3d3_480f_a771_9e59aedee860.slice/crio-ab6782be5d63d2c0d9c38a574eb5c57b3c672f3f18c5d50a6ea9df71d8e6e041 WatchSource:0}: Error finding container ab6782be5d63d2c0d9c38a574eb5c57b3c672f3f18c5d50a6ea9df71d8e6e041: Status 404 returned error can't find the container with id ab6782be5d63d2c0d9c38a574eb5c57b3c672f3f18c5d50a6ea9df71d8e6e041
Sep 15 12:38:25.547998 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.547958    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:25.548109 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.548028    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:25.548109 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.548062    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:25.601640 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.601613    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"]
Sep 15 12:38:25.605897 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:38:25.605875    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod229379c4_14a8_411f_84fb_bb5d2b7dbca6.slice/crio-a11f4e9a76c4521fb3a46606b042d512cc034751807cc46fba3fc0cddf1c656c WatchSource:0}: Error finding container a11f4e9a76c4521fb3a46606b042d512cc034751807cc46fba3fc0cddf1c656c: Status 404 returned error can't find the container with id a11f4e9a76c4521fb3a46606b042d512cc034751807cc46fba3fc0cddf1c656c
Sep 15 12:38:25.608885 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.608850    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:25.608968 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.608940    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:25.609011 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:25.609000    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:26.174771 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:26.174733    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" event={"ID":"229379c4-14a8-411f-84fb-bb5d2b7dbca6","Type":"ContainerStarted","Data":"36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c"}
Sep 15 12:38:26.175210 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:26.174833    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" event={"ID":"229379c4-14a8-411f-84fb-bb5d2b7dbca6","Type":"ContainerStarted","Data":"a11f4e9a76c4521fb3a46606b042d512cc034751807cc46fba3fc0cddf1c656c"}
Sep 15 12:38:26.176506 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:26.176481    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" event={"ID":"70eca63a-d3d3-480f-a771-9e59aedee860","Type":"ContainerStarted","Data":"a0ae41559f2523cdfd14cef642386c22a7294b808de4eca296494dec7d7112ca"}
Sep 15 12:38:26.176644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:26.176517    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" event={"ID":"70eca63a-d3d3-480f-a771-9e59aedee860","Type":"ContainerStarted","Data":"ab6782be5d63d2c0d9c38a574eb5c57b3c672f3f18c5d50a6ea9df71d8e6e041"}
Sep 15 12:38:26.198018 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:26.197962    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" podStartSLOduration=2.197943587 podStartE2EDuration="2.197943587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:38:26.195911662 +0000 UTC m=+774.007823104" watchObservedRunningTime="2026-09-15 12:38:26.197943587 +0000 UTC m=+774.009855030"
Sep 15 12:38:26.218421 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:26.218377    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" podStartSLOduration=2.218359058 podStartE2EDuration="2.218359058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:38:26.217281327 +0000 UTC m=+774.029192782" watchObservedRunningTime="2026-09-15 12:38:26.218359058 +0000 UTC m=+774.030270502"
Sep 15 12:38:26.268756 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:26.268733    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:26.273846 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:26.273789    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:26.322692 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:26.322665    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:26.327271 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:26.327243    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:27.182069 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:27.182040    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:27.182468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:27.182227    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:27.182986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:27.182968    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:27.183103 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:27.183014    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:29.359901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.359869    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"]
Sep 15 12:38:29.363867 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.363842    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.365988 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.365968    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-11f865-serving-cert\""
Sep 15 12:38:29.373323 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.373133    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"]
Sep 15 12:38:29.484948 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.484909    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6dr8\" (UniqueName: \"kubernetes.io/projected/25994587-5d81-4602-8242-29fb1e013160-kube-api-access-v6dr8\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.485117 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.484962    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-metrics-certs\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.485117 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.485018    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25994587-5d81-4602-8242-29fb1e013160-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.485117 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.485088    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-maas-api-tls\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.586209 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.586172    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6dr8\" (UniqueName: \"kubernetes.io/projected/25994587-5d81-4602-8242-29fb1e013160-kube-api-access-v6dr8\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.586398 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.586229    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-metrics-certs\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.586398 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.586263    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25994587-5d81-4602-8242-29fb1e013160-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.586502 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.586447    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-maas-api-tls\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.586993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.586962    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25994587-5d81-4602-8242-29fb1e013160-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.588695 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.588676    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-metrics-certs\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.588809 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.588721    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-maas-api-tls\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.595261 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.595236    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6dr8\" (UniqueName: \"kubernetes.io/projected/25994587-5d81-4602-8242-29fb1e013160-kube-api-access-v6dr8\") pod \"maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.677405 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.677305    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:29.957383 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:29.957359    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"]
Sep 15 12:38:29.959969 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:38:29.959943    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25994587_5d81_4602_8242_29fb1e013160.slice/crio-ff527fc91dd3c8a0b737b344c9a0a4c69487c3f9f5f05a8ea604b1aa91477c28 WatchSource:0}: Error finding container ff527fc91dd3c8a0b737b344c9a0a4c69487c3f9f5f05a8ea604b1aa91477c28: Status 404 returned error can't find the container with id ff527fc91dd3c8a0b737b344c9a0a4c69487c3f9f5f05a8ea604b1aa91477c28
Sep 15 12:38:30.200650 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:30.200602    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f" event={"ID":"25994587-5d81-4602-8242-29fb1e013160","Type":"ContainerStarted","Data":"ff527fc91dd3c8a0b737b344c9a0a4c69487c3f9f5f05a8ea604b1aa91477c28"}
Sep 15 12:38:31.208635 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.208597    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f" event={"ID":"25994587-5d81-4602-8242-29fb1e013160","Type":"ContainerStarted","Data":"3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c"}
Sep 15 12:38:31.209049 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.208761    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:31.225392 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.225348    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f" podStartSLOduration=1.7433413789999999 podStartE2EDuration="2.225334634s" totalImagesPullingTime="481.993255ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:38:31.223429955 +0000 UTC m=+779.035341411" watchObservedRunningTime="2026-09-15 12:38:31.225334634 +0000 UTC m=+779.037246074"
Sep 15 12:38:31.341988 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.341952    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"]
Sep 15 12:38:31.346150 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.346129    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.350384 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.350363    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-ee96ec-serving-cert\""
Sep 15 12:38:31.361767 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.361742    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"]
Sep 15 12:38:31.505397 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.505365    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-metrics-certs\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.505561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.505431    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.505561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.505459    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67e07c56-f32e-47b7-aadc-467bc900eb14-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.505561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.505483    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drwbv\" (UniqueName: \"kubernetes.io/projected/67e07c56-f32e-47b7-aadc-467bc900eb14-kube-api-access-drwbv\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.606985 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.606954    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-metrics-certs\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.607172 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.607012    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.607172 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.607044    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67e07c56-f32e-47b7-aadc-467bc900eb14-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.607172 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.607067    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drwbv\" (UniqueName: \"kubernetes.io/projected/67e07c56-f32e-47b7-aadc-467bc900eb14-kube-api-access-drwbv\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.607821 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.607767    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67e07c56-f32e-47b7-aadc-467bc900eb14-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.609455 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.609434    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-metrics-certs\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.609531 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.609482    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.615702 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.615681    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drwbv\" (UniqueName: \"kubernetes.io/projected/67e07c56-f32e-47b7-aadc-467bc900eb14-kube-api-access-drwbv\") pod \"maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.657473 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.657443    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:31.937883 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:31.937762    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"]
Sep 15 12:38:31.940457 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:38:31.940428    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e07c56_f32e_47b7_aadc_467bc900eb14.slice/crio-c8f9027f7e5fae3b244bcc860a7d39202136df5cc3d327a51d53ed88c5f2053e WatchSource:0}: Error finding container c8f9027f7e5fae3b244bcc860a7d39202136df5cc3d327a51d53ed88c5f2053e: Status 404 returned error can't find the container with id c8f9027f7e5fae3b244bcc860a7d39202136df5cc3d327a51d53ed88c5f2053e
Sep 15 12:38:32.215814 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.215711    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm" event={"ID":"67e07c56-f32e-47b7-aadc-467bc900eb14","Type":"ContainerStarted","Data":"c8f9027f7e5fae3b244bcc860a7d39202136df5cc3d327a51d53ed88c5f2053e"}
Sep 15 12:38:32.355651 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.355623    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"]
Sep 15 12:38:32.359650 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.359624    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.362040 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.362014    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4609ba-serving-cert\""
Sep 15 12:38:32.369991 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.369968    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"]
Sep 15 12:38:32.517011 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.516976    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/802ad009-5cec-4276-a9f5-8789c6dec747-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.517166 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.517030    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-maas-api-tls\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.517166 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.517067    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bp7q\" (UniqueName: \"kubernetes.io/projected/802ad009-5cec-4276-a9f5-8789c6dec747-kube-api-access-4bp7q\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.517166 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.517146    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-metrics-certs\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.617937 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.617896    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/802ad009-5cec-4276-a9f5-8789c6dec747-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.618075 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.617954    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-maas-api-tls\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.618075 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.617990    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bp7q\" (UniqueName: \"kubernetes.io/projected/802ad009-5cec-4276-a9f5-8789c6dec747-kube-api-access-4bp7q\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.618075 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.618032    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-metrics-certs\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.618599 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.618575    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/802ad009-5cec-4276-a9f5-8789c6dec747-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.620289 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.620262    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-metrics-certs\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.620289 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.620284    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4609ba-serving-cert\""
Sep 15 12:38:32.625519 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:32.625497    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bp7q\" (UniqueName: \"kubernetes.io/projected/802ad009-5cec-4276-a9f5-8789c6dec747-kube-api-access-4bp7q\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:32.628300 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:38:32.628279    2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-serving-cert: secret "maas-api-e2e-inf-a-4609ba-serving-cert" not found
Sep 15 12:38:32.628413 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:38:32.628344    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-maas-api-tls podName:802ad009-5cec-4276-a9f5-8789c6dec747 nodeName:}" failed. No retries permitted until 2026-09-15 12:38:33.128325397 +0000 UTC m=+780.940236826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-maas-api-tls") pod "maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm" (UID: "802ad009-5cec-4276-a9f5-8789c6dec747") : secret "maas-api-e2e-inf-a-4609ba-serving-cert" not found
Sep 15 12:38:33.177358 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.177323    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"]
Sep 15 12:38:33.182620 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.182597    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.185218 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.185186    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-bf37f4-openshift-default-dockercfg-rpbbx\""
Sep 15 12:38:33.191999 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.191975    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"]
Sep 15 12:38:33.222818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.222761    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-maas-api-tls\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:33.223996 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.223960    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm" event={"ID":"67e07c56-f32e-47b7-aadc-467bc900eb14","Type":"ContainerStarted","Data":"5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47"}
Sep 15 12:38:33.224132 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.224076    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:33.225234 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.225217    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-maas-api-tls\") pod \"maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:33.242421 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.242374    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm" podStartSLOduration=1.92015721 podStartE2EDuration="2.242358267s" totalImagesPullingTime="322.201057ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:38:33.242003996 +0000 UTC m=+781.053915443" watchObservedRunningTime="2026-09-15 12:38:33.242358267 +0000 UTC m=+781.054269735"
Sep 15 12:38:33.274445 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.274415    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:33.324561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.323671    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-data\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.324561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.323735    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-podinfo\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.324561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.323800    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/480d35db-c411-41d1-815c-5dbb1e69a65b-istiod-ca-cert\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.324561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.323838    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-envoy\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.324561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.323905    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-certs\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.324561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.324018    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-credential-socket\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.324561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.324067    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-token\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.324561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.324105    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5d76\" (UniqueName: \"kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-kube-api-access-g5d76\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.324561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.324166    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-socket\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425170 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425133    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/480d35db-c411-41d1-815c-5dbb1e69a65b-istiod-ca-cert\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425361 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425191    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-envoy\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425361 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425310    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-certs\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425482 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425400    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-credential-socket\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425482 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425436    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-token\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425482 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425458    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5d76\" (UniqueName: \"kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-kube-api-access-g5d76\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425632 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425492    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-socket\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425632 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425538    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-data\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425632 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425558    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-podinfo\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425802 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425670    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-certs\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425802 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425708    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-credential-socket\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.425969 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425936    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/480d35db-c411-41d1-815c-5dbb1e69a65b-istiod-ca-cert\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.426034 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425979    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-data\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.426034 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.425983    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-socket\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.427593 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.427542    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-envoy\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.427853 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.427837    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-podinfo\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.433192 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.433171    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-token\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.433386 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.433367    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5d76\" (UniqueName: \"kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-kube-api-access-g5d76\") pod \"e2e-autores-bf37f4-openshift-default-8664fd7466-69gln\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") " pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.494808 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.494755    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:33.551924 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.551847    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"]
Sep 15 12:38:33.554253 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:38:33.554224    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod802ad009_5cec_4276_a9f5_8789c6dec747.slice/crio-990b9ae5fd6899415c1d8237d5b900b295006c8f5711a8a983b986c62f15c2e2 WatchSource:0}: Error finding container 990b9ae5fd6899415c1d8237d5b900b295006c8f5711a8a983b986c62f15c2e2: Status 404 returned error can't find the container with id 990b9ae5fd6899415c1d8237d5b900b295006c8f5711a8a983b986c62f15c2e2
Sep 15 12:38:33.770900 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.770870    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"]
Sep 15 12:38:33.772616 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:38:33.772587    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod480d35db_c411_41d1_815c_5dbb1e69a65b.slice/crio-2056fad6948d25f43579ce3f7f57e8857dc1afb96bbe38143ea8ff4f631cd3ff WatchSource:0}: Error finding container 2056fad6948d25f43579ce3f7f57e8857dc1afb96bbe38143ea8ff4f631cd3ff: Status 404 returned error can't find the container with id 2056fad6948d25f43579ce3f7f57e8857dc1afb96bbe38143ea8ff4f631cd3ff
Sep 15 12:38:33.775012 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.774979    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:33.775105 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.775047    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:33.775105 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:33.775081    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:34.230734 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.230633    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" event={"ID":"480d35db-c411-41d1-815c-5dbb1e69a65b","Type":"ContainerStarted","Data":"d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317"}
Sep 15 12:38:34.230734 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.230704    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" event={"ID":"480d35db-c411-41d1-815c-5dbb1e69a65b","Type":"ContainerStarted","Data":"2056fad6948d25f43579ce3f7f57e8857dc1afb96bbe38143ea8ff4f631cd3ff"}
Sep 15 12:38:34.232426 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.232397    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm" event={"ID":"802ad009-5cec-4276-a9f5-8789c6dec747","Type":"ContainerStarted","Data":"f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613"}
Sep 15 12:38:34.232560 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.232452    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm" event={"ID":"802ad009-5cec-4276-a9f5-8789c6dec747","Type":"ContainerStarted","Data":"990b9ae5fd6899415c1d8237d5b900b295006c8f5711a8a983b986c62f15c2e2"}
Sep 15 12:38:34.232560 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.232500    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:34.255297 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.255246    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" podStartSLOduration=1.255230986 podStartE2EDuration="1.255230986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:38:34.253331992 +0000 UTC m=+782.065243438" watchObservedRunningTime="2026-09-15 12:38:34.255230986 +0000 UTC m=+782.067142427"
Sep 15 12:38:34.276484 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.276441    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm" podStartSLOduration=1.921602043 podStartE2EDuration="2.276426472s" totalImagesPullingTime="354.824429ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:38:34.274498583 +0000 UTC m=+782.086410024" watchObservedRunningTime="2026-09-15 12:38:34.276426472 +0000 UTC m=+782.088337913"
Sep 15 12:38:34.495959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.495923    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:34.501550 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.501525    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:34.815698 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.815617    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"]
Sep 15 12:38:34.819124 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.819099    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:34.821912 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.821888    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1f5381de-openshift-default-dockercfg-82mgw\""
Sep 15 12:38:34.844709 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.844686    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"]
Sep 15 12:38:34.942699 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.942666    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istiod-ca-cert\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:34.942699 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.942717    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-envoy\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:34.942982 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.942803    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-credential-socket\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:34.942982 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.942836    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-podinfo\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:34.942982 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.942924    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-certs\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:34.943315 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.942965    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-socket\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:34.943315 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.943088    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9xrs\" (UniqueName: \"kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-kube-api-access-w9xrs\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:34.943315 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.943119    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-data\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:34.943315 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:34.943149    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-token\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044090 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044060    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-credential-socket\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044090 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044094    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-podinfo\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044302 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044122    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-certs\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044302 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044137    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-socket\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044302 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044191    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9xrs\" (UniqueName: \"kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-kube-api-access-w9xrs\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044302 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044213    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-data\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044302 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044244    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-token\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044556 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044316    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istiod-ca-cert\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044556 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044398    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-envoy\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044556 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044515    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-credential-socket\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044717 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044683    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-socket\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044773 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044716    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-certs\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.044773 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.044737    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-data\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.045120 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.045097    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istiod-ca-cert\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.046494 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.046476    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-envoy\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.046753 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.046734    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-podinfo\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.060736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.060708    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-token\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.060933 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.060915    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9xrs\" (UniqueName: \"kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-kube-api-access-w9xrs\") pod \"e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") " pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.130724 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.130651    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:35.238014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.237982    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:35.238949 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.238932    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:38:35.411966 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.411944    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"]
Sep 15 12:38:35.414651 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:38:35.414616    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe5404e_f219_4c1a_ab03_5c9d1ec98706.slice/crio-e2bc94f3cd3768aeecc2081f944eeb0777d7ecb95888ba7e3625f0be9bd28581 WatchSource:0}: Error finding container e2bc94f3cd3768aeecc2081f944eeb0777d7ecb95888ba7e3625f0be9bd28581: Status 404 returned error can't find the container with id e2bc94f3cd3768aeecc2081f944eeb0777d7ecb95888ba7e3625f0be9bd28581
Sep 15 12:38:35.417076 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.417040    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:35.417184 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.417109    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:35.417184 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:35.417135    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:36.249984 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:36.249949    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" event={"ID":"2fe5404e-f219-4c1a-ab03-5c9d1ec98706","Type":"ContainerStarted","Data":"247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869"}
Sep 15 12:38:36.250614 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:36.250503    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" event={"ID":"2fe5404e-f219-4c1a-ab03-5c9d1ec98706","Type":"ContainerStarted","Data":"e2bc94f3cd3768aeecc2081f944eeb0777d7ecb95888ba7e3625f0be9bd28581"}
Sep 15 12:38:36.275794 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:36.275734    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" podStartSLOduration=2.275713476 podStartE2EDuration="2.275713476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:38:36.273928625 +0000 UTC m=+784.085840067" watchObservedRunningTime="2026-09-15 12:38:36.275713476 +0000 UTC m=+784.087624918"
Sep 15 12:38:37.131107 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:37.131074    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:37.136239 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:37.136214    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:37.221887 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:37.221863    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:38:37.255170 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:37.255097    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:37.256904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:37.256811    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:38:39.246116 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:39.246076    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:38:40.251977 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:40.251944    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:38:42.572207 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.572175    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"]
Sep 15 12:38:42.576491 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.576469    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:38:42.579091 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.579067    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1f5381de-dockercfg-j7bnq\""
Sep 15 12:38:42.579604 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.579550    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1f5381de\""
Sep 15 12:38:42.586435 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.586413    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"]
Sep 15 12:38:42.665629 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.665601    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"]
Sep 15 12:38:42.669116 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.669098    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:38:42.679110 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.678507    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"]
Sep 15 12:38:42.765983 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.765930    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds\" (UID: \"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:38:42.766305 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.766280    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-plugins-config-volume\") pod \"payload-processing-e2e-ait-1f5381de-59f479465-f96p7\" (UID: \"001f2f5d-e803-4f8a-acd6-f3f8d12476ff\") " pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:38:42.767204 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.767185    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gxzs\" (UniqueName: \"kubernetes.io/projected/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-kube-api-access-8gxzs\") pod \"payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds\" (UID: \"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:38:42.767458 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.767438    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m9ck\" (UniqueName: \"kubernetes.io/projected/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-kube-api-access-2m9ck\") pod \"payload-processing-e2e-ait-1f5381de-59f479465-f96p7\" (UID: \"001f2f5d-e803-4f8a-acd6-f3f8d12476ff\") " pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:38:42.869216 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.869123    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds\" (UID: \"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:38:42.869893 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.869834    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds\" (UID: \"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:38:42.869962 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.869882    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-plugins-config-volume\") pod \"payload-processing-e2e-ait-1f5381de-59f479465-f96p7\" (UID: \"001f2f5d-e803-4f8a-acd6-f3f8d12476ff\") " pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:38:42.870058 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.870038    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gxzs\" (UniqueName: \"kubernetes.io/projected/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-kube-api-access-8gxzs\") pod \"payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds\" (UID: \"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:38:42.870110 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.870074    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m9ck\" (UniqueName: \"kubernetes.io/projected/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-kube-api-access-2m9ck\") pod \"payload-processing-e2e-ait-1f5381de-59f479465-f96p7\" (UID: \"001f2f5d-e803-4f8a-acd6-f3f8d12476ff\") " pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:38:42.870592 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.870568    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-plugins-config-volume\") pod \"payload-processing-e2e-ait-1f5381de-59f479465-f96p7\" (UID: \"001f2f5d-e803-4f8a-acd6-f3f8d12476ff\") " pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:38:42.878304 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.878287    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gxzs\" (UniqueName: \"kubernetes.io/projected/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-kube-api-access-8gxzs\") pod \"payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds\" (UID: \"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:38:42.878431 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.878302    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m9ck\" (UniqueName: \"kubernetes.io/projected/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-kube-api-access-2m9ck\") pod \"payload-processing-e2e-ait-1f5381de-59f479465-f96p7\" (UID: \"001f2f5d-e803-4f8a-acd6-f3f8d12476ff\") " pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:38:42.889145 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.889126    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:38:42.982446 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:42.982405    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:38:43.264710 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:43.257251    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"]
Sep 15 12:38:43.270804 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:38:43.270258    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc75f446c_ecae_4bc6_b4d6_c5d96ec9f654.slice/crio-c220b0a302398f760978cb303bf555ee998d3b8a7eccaef873a6ba86ff580610 WatchSource:0}: Error finding container c220b0a302398f760978cb303bf555ee998d3b8a7eccaef873a6ba86ff580610: Status 404 returned error can't find the container with id c220b0a302398f760978cb303bf555ee998d3b8a7eccaef873a6ba86ff580610
Sep 15 12:38:43.290719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:43.290689    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds" event={"ID":"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654","Type":"ContainerStarted","Data":"c220b0a302398f760978cb303bf555ee998d3b8a7eccaef873a6ba86ff580610"}
Sep 15 12:38:43.392111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:43.392078    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"]
Sep 15 12:38:43.394004 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:38:43.393862    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod001f2f5d_e803_4f8a_acd6_f3f8d12476ff.slice/crio-dda27c6a1fb9c5f084b90af505296254e6cd6292612e8ea952184c251394c3f4 WatchSource:0}: Error finding container dda27c6a1fb9c5f084b90af505296254e6cd6292612e8ea952184c251394c3f4: Status 404 returned error can't find the container with id dda27c6a1fb9c5f084b90af505296254e6cd6292612e8ea952184c251394c3f4
Sep 15 12:38:44.299460 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:44.298395    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds" event={"ID":"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654","Type":"ContainerStarted","Data":"83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8"}
Sep 15 12:38:44.299460 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:44.299373    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:38:44.302710 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:44.302200    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7" event={"ID":"001f2f5d-e803-4f8a-acd6-f3f8d12476ff","Type":"ContainerStarted","Data":"7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6"}
Sep 15 12:38:44.302710 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:44.302250    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7" event={"ID":"001f2f5d-e803-4f8a-acd6-f3f8d12476ff","Type":"ContainerStarted","Data":"dda27c6a1fb9c5f084b90af505296254e6cd6292612e8ea952184c251394c3f4"}
Sep 15 12:38:44.302710 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:44.302640    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:38:44.335896 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:44.335800    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds" podStartSLOduration=2.3357608819999998 podStartE2EDuration="2.335760882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:38:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:38:44.318649014 +0000 UTC m=+792.130560486" watchObservedRunningTime="2026-09-15 12:38:44.335760882 +0000 UTC m=+792.147672326"
Sep 15 12:38:44.338111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:44.338066    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7" podStartSLOduration=2.338053456 podStartE2EDuration="2.338053456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:38:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:38:44.33490041 +0000 UTC m=+792.146811865" watchObservedRunningTime="2026-09-15 12:38:44.338053456 +0000 UTC m=+792.149964899"
Sep 15 12:38:47.323128 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:47.323052    2586 generic.go:356] "Generic (PLEG): container finished" podID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerID="36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c" exitCode=0
Sep 15 12:38:47.323128 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:47.323099    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" event={"ID":"229379c4-14a8-411f-84fb-bb5d2b7dbca6","Type":"ContainerDied","Data":"36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c"}
Sep 15 12:38:47.324662 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:47.323557    2586 scope.go:117] "RemoveContainer" containerID="36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c"
Sep 15 12:38:47.325087 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:47.325025    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:47.325148 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:47.325119    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:47.325200 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:47.325179    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:48.340476 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:48.340439    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" event={"ID":"229379c4-14a8-411f-84fb-bb5d2b7dbca6","Type":"ContainerStarted","Data":"d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1"}
Sep 15 12:38:48.342538 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:48.342508    2586 generic.go:356] "Generic (PLEG): container finished" podID="70eca63a-d3d3-480f-a771-9e59aedee860" containerID="a0ae41559f2523cdfd14cef642386c22a7294b808de4eca296494dec7d7112ca" exitCode=0
Sep 15 12:38:48.342653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:48.342641    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" event={"ID":"70eca63a-d3d3-480f-a771-9e59aedee860","Type":"ContainerDied","Data":"a0ae41559f2523cdfd14cef642386c22a7294b808de4eca296494dec7d7112ca"}
Sep 15 12:38:48.342993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:48.342979    2586 scope.go:117] "RemoveContainer" containerID="a0ae41559f2523cdfd14cef642386c22a7294b808de4eca296494dec7d7112ca"
Sep 15 12:38:48.344084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:48.344047    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:48.344178 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:48.344123    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:48.344178 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:48.344158    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:49.322748 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:49.322714    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:49.323435 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:49.323398    2586 patch_prober.go:28] interesting pod/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:38:49.323567 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:49.323460    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 12:38:49.355800 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:49.355741    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" event={"ID":"70eca63a-d3d3-480f-a771-9e59aedee860","Type":"ContainerStarted","Data":"80220efd626e953268da06f4f1ee37f4df06417f49f73567b24adabeffbd2c2b"}
Sep 15 12:38:50.269157 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:50.269123    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:50.269528 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:50.269496    2586 patch_prober.go:28] interesting pod/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:38:50.269648 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:50.269559    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 12:38:50.325327 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:50.325282    2586 patch_prober.go:28] interesting pod/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:38:50.325619 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:50.325570    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 12:38:51.274468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:51.274421    2586 patch_prober.go:28] interesting pod/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:38:51.275007 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:51.274491    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 12:38:51.325191 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:51.325148    2586 patch_prober.go:28] interesting pod/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:38:51.325360 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:51.325213    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 12:38:52.269636 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:52.269172    2586 patch_prober.go:28] interesting pod/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:38:52.269636 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:52.269250    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 12:38:52.322210 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:52.322173    2586 patch_prober.go:28] interesting pod/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:38:52.322862 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:52.322828    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 12:38:53.269671 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.269631    2586 patch_prober.go:28] interesting pod/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:38:53.269990 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.269949    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 12:38:53.323067 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.323031    2586 patch_prober.go:28] interesting pod/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:38:53.323671 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.323623    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 12:38:53.758130 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.758082    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"]
Sep 15 12:38:53.769171 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.769146    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.773126 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.772933    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-2e3fef-openshift-default-dockercfg-p79hr\""
Sep 15 12:38:53.777285 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.777264    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"]
Sep 15 12:38:53.814827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.813316    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-socket\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.814827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.813408    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-credential-socket\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.814827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.813445    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff85a67-fbc8-4f77-acf4-33305db8e41e-istiod-ca-cert\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.814827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.813476    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-envoy\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.814827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.813537    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-certs\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.814827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.813573    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-podinfo\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.814827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.813605    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-data\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.814827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.813649    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl7c8\" (UniqueName: \"kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-kube-api-access-dl7c8\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.814827 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.813699    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-token\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.918806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.917352    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-certs\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.918806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.917414    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-podinfo\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.918806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.917453    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-data\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.918806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.917499    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl7c8\" (UniqueName: \"kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-kube-api-access-dl7c8\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.918806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.917551    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-token\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.918806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.917585    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-socket\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.918806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.917673    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-credential-socket\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.918806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.917706    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff85a67-fbc8-4f77-acf4-33305db8e41e-istiod-ca-cert\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.918806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.917737    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-envoy\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.920059 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.920027    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-certs\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.920284 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.920261    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-credential-socket\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.933956 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.930271    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-data\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.933956 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.932100    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl7c8\" (UniqueName: \"kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-kube-api-access-dl7c8\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.933956 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.933053    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-envoy\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.934550 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.934528    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff85a67-fbc8-4f77-acf4-33305db8e41e-istiod-ca-cert\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.941163 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.935011    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-socket\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.943796 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.941493    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-podinfo\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:53.947259 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:53.947234    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-token\") pod \"e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") " pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:54.110129 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:54.110041    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:54.269426 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:54.268936    2586 patch_prober.go:28] interesting pod/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:38:54.269426 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:54.269024    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 12:38:54.324842 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:54.324261    2586 patch_prober.go:28] interesting pod/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:38:54.324842 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:54.324318    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 12:38:54.579111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:54.574827    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"]
Sep 15 12:38:54.607005 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:54.606448    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:54.607005 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:54.606538    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:54.607005 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:54.606582    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:55.269677 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:55.269634    2586 patch_prober.go:28] interesting pod/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:38:55.269871 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:55.269700    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 12:38:55.312850 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:55.312816    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:38:55.334408 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:55.334372    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:55.335464 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:55.335439    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:55.343420 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:55.343392    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:38:55.472414 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:55.461374    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" event={"ID":"bff85a67-fbc8-4f77-acf4-33305db8e41e","Type":"ContainerStarted","Data":"1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3"}
Sep 15 12:38:55.472414 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:55.461430    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" event={"ID":"bff85a67-fbc8-4f77-acf4-33305db8e41e","Type":"ContainerStarted","Data":"098e70d4f043f28604e5dd30f236414e1005ba502ea89bf99e2420eae9b1e8dd"}
Sep 15 12:38:55.504802 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:55.501375    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" podStartSLOduration=2.501351432 podStartE2EDuration="2.501351432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:38:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:38:55.50007513 +0000 UTC m=+803.311986572" watchObservedRunningTime="2026-09-15 12:38:55.501351432 +0000 UTC m=+803.313262875"
Sep 15 12:38:56.113296 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:56.113254    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:56.123074 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:56.123038    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:56.273940 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:56.273905    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:56.274326 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:56.274304    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:56.275150 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:56.275126    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:38:56.322023 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:56.321991    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:38:56.467060 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:56.466924    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:56.467997 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:56.467974    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:38:59.488130 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:59.488100    2586 generic.go:356] "Generic (PLEG): container finished" podID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerID="d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317" exitCode=0
Sep 15 12:38:59.488523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:59.488189    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" event={"ID":"480d35db-c411-41d1-815c-5dbb1e69a65b","Type":"ContainerDied","Data":"d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317"}
Sep 15 12:38:59.488817 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:59.488795    2586 scope.go:117] "RemoveContainer" containerID="d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317"
Sep 15 12:38:59.490189 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:59.490157    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:59.490306 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:59.490247    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:38:59.490306 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:38:59.490299    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:00.496229 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:00.496193    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" event={"ID":"480d35db-c411-41d1-815c-5dbb1e69a65b","Type":"ContainerStarted","Data":"5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7"}
Sep 15 12:39:00.498219 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:00.498193    2586 generic.go:356] "Generic (PLEG): container finished" podID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerID="247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869" exitCode=0
Sep 15 12:39:00.498353 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:00.498267    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" event={"ID":"2fe5404e-f219-4c1a-ab03-5c9d1ec98706","Type":"ContainerDied","Data":"247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869"}
Sep 15 12:39:00.498765 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:00.498748    2586 scope.go:117] "RemoveContainer" containerID="247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869"
Sep 15 12:39:00.500084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:00.500055    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:00.500187 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:00.500142    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:00.500254 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:00.500194    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:01.496962 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:01.496913    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:39:01.497512 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:01.497133    2586 patch_prober.go:28] interesting pod/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 12:39:01.497512 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:01.497184    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 12:39:01.511470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:01.511430    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" event={"ID":"2fe5404e-f219-4c1a-ab03-5c9d1ec98706","Type":"ContainerStarted","Data":"9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901"}
Sep 15 12:39:02.131615 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:02.131574    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:39:02.131840 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:02.131753    2586 patch_prober.go:28] interesting pod/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 15 12:39:02.131840 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:02.131819    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 15 12:39:02.502810 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:02.500153    2586 patch_prober.go:28] interesting pod/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 12:39:02.502810 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:02.500217    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 12:39:03.131495 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:03.131457    2586 patch_prober.go:28] interesting pod/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 15 12:39:03.131696 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:03.131528    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 15 12:39:03.496225 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:03.496181    2586 patch_prober.go:28] interesting pod/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 12:39:03.496416 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:03.496253    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 12:39:04.131700 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:04.131661    2586 patch_prober.go:28] interesting pod/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 15 12:39:04.132356 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:04.132307    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 15 12:39:04.495484 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:04.495452    2586 patch_prober.go:28] interesting pod/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 12:39:04.495756 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:04.495731    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 12:39:05.132311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:05.132274    2586 patch_prober.go:28] interesting pod/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 15 12:39:05.132907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:05.132879    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 15 12:39:05.239510 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:05.238209    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:39:05.496381 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:05.495936    2586 patch_prober.go:28] interesting pod/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 15 12:39:05.496381 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:05.496003    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 15 12:39:06.132068 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:06.132027    2586 patch_prober.go:28] interesting pod/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 15 12:39:06.132220 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:06.132085    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 15 12:39:06.512088 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:06.512051    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:39:06.517574 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:06.517543    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:39:07.135738 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:07.135704    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:39:07.136071 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:07.136044    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:39:07.136708 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:07.136691    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:39:08.635883 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:08.635850    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"]
Sep 15 12:39:08.636655 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:08.636618    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7" podUID="001f2f5d-e803-4f8a-acd6-f3f8d12476ff" containerName="payload-processing" containerID="cri-o://7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6" gracePeriod=30
Sep 15 12:39:08.657683 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:08.657651    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"]
Sep 15 12:39:08.657946 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:08.657897    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds" podUID="c75f446c-ecae-4bc6-b4d6-c5d96ec9f654" containerName="payload-pre-processing" containerID="cri-o://83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8" gracePeriod=30
Sep 15 12:39:08.914487 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:08.914464    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:39:08.957645 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:08.957617    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"]
Sep 15 12:39:08.959574 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:08.959553    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:39:09.050965 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.050934    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-plugins-config-volume\") pod \"001f2f5d-e803-4f8a-acd6-f3f8d12476ff\" (UID: \"001f2f5d-e803-4f8a-acd6-f3f8d12476ff\") "
Sep 15 12:39:09.051124 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.050974    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m9ck\" (UniqueName: \"kubernetes.io/projected/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-kube-api-access-2m9ck\") pod \"001f2f5d-e803-4f8a-acd6-f3f8d12476ff\" (UID: \"001f2f5d-e803-4f8a-acd6-f3f8d12476ff\") "
Sep 15 12:39:09.051124 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.051068    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gxzs\" (UniqueName: \"kubernetes.io/projected/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-kube-api-access-8gxzs\") pod \"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654\" (UID: \"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654\") "
Sep 15 12:39:09.051286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.051137    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-plugins-config-volume\") pod \"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654\" (UID: \"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654\") "
Sep 15 12:39:09.051401 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.051374    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "001f2f5d-e803-4f8a-acd6-f3f8d12476ff" (UID: "001f2f5d-e803-4f8a-acd6-f3f8d12476ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:39:09.051530 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.051515    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:09.051576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.051510    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c75f446c-ecae-4bc6-b4d6-c5d96ec9f654" (UID: "c75f446c-ecae-4bc6-b4d6-c5d96ec9f654"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:39:09.053215 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.053193    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-kube-api-access-8gxzs" (OuterVolumeSpecName: "kube-api-access-8gxzs") pod "c75f446c-ecae-4bc6-b4d6-c5d96ec9f654" (UID: "c75f446c-ecae-4bc6-b4d6-c5d96ec9f654"). InnerVolumeSpecName "kube-api-access-8gxzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:39:09.053304 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.053288    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-kube-api-access-2m9ck" (OuterVolumeSpecName: "kube-api-access-2m9ck") pod "001f2f5d-e803-4f8a-acd6-f3f8d12476ff" (UID: "001f2f5d-e803-4f8a-acd6-f3f8d12476ff"). InnerVolumeSpecName "kube-api-access-2m9ck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:39:09.153126 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.153047    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:09.153126 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.153073    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m9ck\" (UniqueName: \"kubernetes.io/projected/001f2f5d-e803-4f8a-acd6-f3f8d12476ff-kube-api-access-2m9ck\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:09.153126 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.153082    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gxzs\" (UniqueName: \"kubernetes.io/projected/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654-kube-api-access-8gxzs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:09.582978 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.582943    2586 generic.go:356] "Generic (PLEG): container finished" podID="c75f446c-ecae-4bc6-b4d6-c5d96ec9f654" containerID="83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8" exitCode=0
Sep 15 12:39:09.583163 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.582987    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"
Sep 15 12:39:09.583163 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.583021    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds" event={"ID":"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654","Type":"ContainerDied","Data":"83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8"}
Sep 15 12:39:09.583163 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.583072    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds" event={"ID":"c75f446c-ecae-4bc6-b4d6-c5d96ec9f654","Type":"ContainerDied","Data":"c220b0a302398f760978cb303bf555ee998d3b8a7eccaef873a6ba86ff580610"}
Sep 15 12:39:09.583163 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.583095    2586 scope.go:117] "RemoveContainer" containerID="83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8"
Sep 15 12:39:09.584495 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.584472    2586 generic.go:356] "Generic (PLEG): container finished" podID="001f2f5d-e803-4f8a-acd6-f3f8d12476ff" containerID="7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6" exitCode=0
Sep 15 12:39:09.584625 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.584517    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"
Sep 15 12:39:09.584752 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.584722    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7" event={"ID":"001f2f5d-e803-4f8a-acd6-f3f8d12476ff","Type":"ContainerDied","Data":"7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6"}
Sep 15 12:39:09.584908 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.584807    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7" event={"ID":"001f2f5d-e803-4f8a-acd6-f3f8d12476ff","Type":"ContainerDied","Data":"dda27c6a1fb9c5f084b90af505296254e6cd6292612e8ea952184c251394c3f4"}
Sep 15 12:39:09.584908 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.584727    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy" containerID="cri-o://9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901" gracePeriod=30
Sep 15 12:39:09.592471 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.592419    2586 scope.go:117] "RemoveContainer" containerID="83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8"
Sep 15 12:39:09.592881 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:39:09.592743    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8\": container with ID starting with 83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8 not found: ID does not exist" containerID="83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8"
Sep 15 12:39:09.592881 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.592807    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8"} err="failed to get container status \"83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8\": rpc error: code = NotFound desc = could not find container \"83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8\": container with ID starting with 83b54effe0a6c80310720eec293ead4fa1dc37a1f34432efeb0b80f41505f6d8 not found: ID does not exist"
Sep 15 12:39:09.592881 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.592833    2586 scope.go:117] "RemoveContainer" containerID="7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6"
Sep 15 12:39:09.602213 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.602194    2586 scope.go:117] "RemoveContainer" containerID="7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6"
Sep 15 12:39:09.602652 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:39:09.602634    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6\": container with ID starting with 7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6 not found: ID does not exist" containerID="7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6"
Sep 15 12:39:09.602705 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.602658    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6"} err="failed to get container status \"7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6\": rpc error: code = NotFound desc = could not find container \"7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6\": container with ID starting with 7adb857cabe6af77cbb1a53e9a81f64025397ce6a0df4b002af0e7f5204527e6 not found: ID does not exist"
Sep 15 12:39:09.626626 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.626597    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"]
Sep 15 12:39:09.632483 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.632462    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f5381de-59f479465-f96p7"]
Sep 15 12:39:09.644576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.644550    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"]
Sep 15 12:39:09.649508 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.649488    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f5381de-77f5546646-b8dds"]
Sep 15 12:39:09.981974 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.981894    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"]
Sep 15 12:39:09.982468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.982444    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c75f446c-ecae-4bc6-b4d6-c5d96ec9f654" containerName="payload-pre-processing"
Sep 15 12:39:09.982468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.982468    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c75f446c-ecae-4bc6-b4d6-c5d96ec9f654" containerName="payload-pre-processing"
Sep 15 12:39:09.982923 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.982496    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="001f2f5d-e803-4f8a-acd6-f3f8d12476ff" containerName="payload-processing"
Sep 15 12:39:09.982923 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.982505    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="001f2f5d-e803-4f8a-acd6-f3f8d12476ff" containerName="payload-processing"
Sep 15 12:39:09.982923 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.982672    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c75f446c-ecae-4bc6-b4d6-c5d96ec9f654" containerName="payload-pre-processing"
Sep 15 12:39:09.982923 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.982696    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="001f2f5d-e803-4f8a-acd6-f3f8d12476ff" containerName="payload-processing"
Sep 15 12:39:09.985959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.985938    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:09.988648 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.988627    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ecae662b-gw-openshift-default-dockercfg-92mv4\""
Sep 15 12:39:09.998582 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:09.998557    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"]
Sep 15 12:39:10.062759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.062728    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-podinfo\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.062938 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.062830    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-socket\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.062938 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.062854    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-token\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.062938 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.062882    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-envoy\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.062938 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.062900    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-credential-socket\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.063084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.062971    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istiod-ca-cert\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.063084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.063021    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-certs\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.063084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.063064    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-data\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.063084 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.063081    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwx54\" (UniqueName: \"kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-kube-api-access-hwx54\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.163969 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.163938    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-envoy\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164170 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.163974    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-credential-socket\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164170 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164003    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istiod-ca-cert\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164170 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164055    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-certs\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164170 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164094    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-data\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164170 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164117    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwx54\" (UniqueName: \"kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-kube-api-access-hwx54\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164170 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164162    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-podinfo\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164491 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164218    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-socket\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164491 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164242    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-token\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164491 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164452    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-credential-socket\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164646 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164601    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-data\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164845 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164819    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-socket\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.164958 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.164860    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-certs\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.165138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.165116    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istiod-ca-cert\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.166726 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.166707    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-envoy\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.167287 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.167266    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-podinfo\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.172485 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.172465    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwx54\" (UniqueName: \"kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-kube-api-access-hwx54\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.172580 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.172525    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-token\") pod \"e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") " pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.298773 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.298741    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:10.585495 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.585272    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"]
Sep 15 12:39:10.588216 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:39:10.588189    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8c7cfe_0ba1_4045_936f_1f78bd1e8afe.slice/crio-e9b8e7466ab98f8824076dcc30be79bf8248c07b38249e66913631621b185e67 WatchSource:0}: Error finding container e9b8e7466ab98f8824076dcc30be79bf8248c07b38249e66913631621b185e67: Status 404 returned error can't find the container with id e9b8e7466ab98f8824076dcc30be79bf8248c07b38249e66913631621b185e67
Sep 15 12:39:10.590612 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.590577    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:10.590706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.590654    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:10.590706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.590694    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:10.656226 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.656202    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="001f2f5d-e803-4f8a-acd6-f3f8d12476ff" path="/var/lib/kubelet/pods/001f2f5d-e803-4f8a-acd6-f3f8d12476ff/volumes"
Sep 15 12:39:10.656647 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:10.656630    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c75f446c-ecae-4bc6-b4d6-c5d96ec9f654" path="/var/lib/kubelet/pods/c75f446c-ecae-4bc6-b4d6-c5d96ec9f654/volumes"
Sep 15 12:39:11.599551 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:11.599511    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" event={"ID":"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe","Type":"ContainerStarted","Data":"4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47"}
Sep 15 12:39:11.599726 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:11.599572    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" event={"ID":"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe","Type":"ContainerStarted","Data":"e9b8e7466ab98f8824076dcc30be79bf8248c07b38249e66913631621b185e67"}
Sep 15 12:39:11.625200 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:11.625153    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" podStartSLOduration=2.62513635 podStartE2EDuration="2.62513635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:39:11.62256867 +0000 UTC m=+819.434480112" watchObservedRunningTime="2026-09-15 12:39:11.62513635 +0000 UTC m=+819.437047875"
Sep 15 12:39:12.298911 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:12.298875    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:12.304524 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:12.304494    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:12.607211 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:12.607122    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:12.608159 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:12.608138    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:14.860750 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:14.860697    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:39:15.015180 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015147    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-data\") pod \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") "
Sep 15 12:39:15.015376 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015197    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-token\") pod \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") "
Sep 15 12:39:15.015376 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015230    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-socket\") pod \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") "
Sep 15 12:39:15.015376 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015289    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-envoy\") pod \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") "
Sep 15 12:39:15.015376 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015346    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9xrs\" (UniqueName: \"kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-kube-api-access-w9xrs\") pod \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") "
Sep 15 12:39:15.015376 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015371    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istiod-ca-cert\") pod \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") "
Sep 15 12:39:15.015675 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015395    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-credential-socket\") pod \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") "
Sep 15 12:39:15.015675 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015427    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-podinfo\") pod \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") "
Sep 15 12:39:15.015675 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015448    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-data" (OuterVolumeSpecName: "istio-data") pod "2fe5404e-f219-4c1a-ab03-5c9d1ec98706" (UID: "2fe5404e-f219-4c1a-ab03-5c9d1ec98706"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:39:15.015675 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015467    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-certs\") pod \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\" (UID: \"2fe5404e-f219-4c1a-ab03-5c9d1ec98706\") "
Sep 15 12:39:15.015925 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015831    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2fe5404e-f219-4c1a-ab03-5c9d1ec98706" (UID: "2fe5404e-f219-4c1a-ab03-5c9d1ec98706"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:39:15.015982 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015971    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:15.016040 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.015991    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:15.016124 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.016091    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2fe5404e-f219-4c1a-ab03-5c9d1ec98706" (UID: "2fe5404e-f219-4c1a-ab03-5c9d1ec98706"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:39:15.016186 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.016171    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2fe5404e-f219-4c1a-ab03-5c9d1ec98706" (UID: "2fe5404e-f219-4c1a-ab03-5c9d1ec98706"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:39:15.016409 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.016382    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2fe5404e-f219-4c1a-ab03-5c9d1ec98706" (UID: "2fe5404e-f219-4c1a-ab03-5c9d1ec98706"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:39:15.018718 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.018682    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2fe5404e-f219-4c1a-ab03-5c9d1ec98706" (UID: "2fe5404e-f219-4c1a-ab03-5c9d1ec98706"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:39:15.018863 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.018762    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-token" (OuterVolumeSpecName: "istio-token") pod "2fe5404e-f219-4c1a-ab03-5c9d1ec98706" (UID: "2fe5404e-f219-4c1a-ab03-5c9d1ec98706"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:39:15.018933 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.018856    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-kube-api-access-w9xrs" (OuterVolumeSpecName: "kube-api-access-w9xrs") pod "2fe5404e-f219-4c1a-ab03-5c9d1ec98706" (UID: "2fe5404e-f219-4c1a-ab03-5c9d1ec98706"). InnerVolumeSpecName "kube-api-access-w9xrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:39:15.018994 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.018952    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2fe5404e-f219-4c1a-ab03-5c9d1ec98706" (UID: "2fe5404e-f219-4c1a-ab03-5c9d1ec98706"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:39:15.116877 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.116737    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:15.116877 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.116766    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:15.116877 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.116802    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:15.116877 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.116837    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9xrs\" (UniqueName: \"kubernetes.io/projected/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-kube-api-access-w9xrs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:15.116877 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.116848    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:15.116877 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.116857    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:15.116877 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.116864    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fe5404e-f219-4c1a-ab03-5c9d1ec98706-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:15.626874 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.626837    2586 generic.go:356] "Generic (PLEG): container finished" podID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerID="9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901" exitCode=0
Sep 15 12:39:15.627063 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.626903    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"
Sep 15 12:39:15.627063 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.626924    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" event={"ID":"2fe5404e-f219-4c1a-ab03-5c9d1ec98706","Type":"ContainerDied","Data":"9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901"}
Sep 15 12:39:15.627063 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.627021    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx" event={"ID":"2fe5404e-f219-4c1a-ab03-5c9d1ec98706","Type":"ContainerDied","Data":"e2bc94f3cd3768aeecc2081f944eeb0777d7ecb95888ba7e3625f0be9bd28581"}
Sep 15 12:39:15.627063 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.627051    2586 scope.go:117] "RemoveContainer" containerID="9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901"
Sep 15 12:39:15.637236 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.637213    2586 scope.go:117] "RemoveContainer" containerID="247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869"
Sep 15 12:39:15.645909 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.645893    2586 scope.go:117] "RemoveContainer" containerID="9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901"
Sep 15 12:39:15.646189 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:39:15.646170    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901\": container with ID starting with 9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901 not found: ID does not exist" containerID="9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901"
Sep 15 12:39:15.646242 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.646197    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901"} err="failed to get container status \"9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901\": rpc error: code = NotFound desc = could not find container \"9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901\": container with ID starting with 9af0e0735d2c4f632dd71377d76f80c58ee62844b2dca9934a16ecc41ed4d901 not found: ID does not exist"
Sep 15 12:39:15.646242 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.646214    2586 scope.go:117] "RemoveContainer" containerID="247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869"
Sep 15 12:39:15.646484 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:39:15.646466    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869\": container with ID starting with 247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869 not found: ID does not exist" containerID="247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869"
Sep 15 12:39:15.646539 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.646493    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869"} err="failed to get container status \"247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869\": rpc error: code = NotFound desc = could not find container \"247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869\": container with ID starting with 247c36f5a6686fd7344e492633b38b867c4c77aae493d53a3329e67ba0d95869 not found: ID does not exist"
Sep 15 12:39:15.650455 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.650432    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"]
Sep 15 12:39:15.652571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:15.652550    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1f5381de-openshift-default-5c548c4ff8-g6mrx"]
Sep 15 12:39:16.654549 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:16.654499    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" path="/var/lib/kubelet/pods/2fe5404e-f219-4c1a-ab03-5c9d1ec98706/volumes"
Sep 15 12:39:17.214799 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.212845    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"]
Sep 15 12:39:17.214799 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.213684    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy"
Sep 15 12:39:17.214799 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.213706    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy"
Sep 15 12:39:17.214799 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.213757    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy"
Sep 15 12:39:17.214799 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.213766    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy"
Sep 15 12:39:17.214799 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.213896    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy"
Sep 15 12:39:17.214799 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.213910    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fe5404e-f219-4c1a-ab03-5c9d1ec98706" containerName="istio-proxy"
Sep 15 12:39:17.223728 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.222999    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"]
Sep 15 12:39:17.223728 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.223118    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:17.227813 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.227712    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-ecae662b\""
Sep 15 12:39:17.228063 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.228000    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-ecae662b-dockercfg-grh9d\""
Sep 15 12:39:17.350566 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.350346    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks6tk\" (UniqueName: \"kubernetes.io/projected/ac5947ff-b785-4525-a5f5-3653fc91c99f-kube-api-access-ks6tk\") pod \"payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq\" (UID: \"ac5947ff-b785-4525-a5f5-3653fc91c99f\") " pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:17.350566 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.350409    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac5947ff-b785-4525-a5f5-3653fc91c99f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq\" (UID: \"ac5947ff-b785-4525-a5f5-3653fc91c99f\") " pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:17.452196 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.452152    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks6tk\" (UniqueName: \"kubernetes.io/projected/ac5947ff-b785-4525-a5f5-3653fc91c99f-kube-api-access-ks6tk\") pod \"payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq\" (UID: \"ac5947ff-b785-4525-a5f5-3653fc91c99f\") " pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:17.452394 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.452205    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac5947ff-b785-4525-a5f5-3653fc91c99f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq\" (UID: \"ac5947ff-b785-4525-a5f5-3653fc91c99f\") " pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:17.452918 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.452896    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac5947ff-b785-4525-a5f5-3653fc91c99f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq\" (UID: \"ac5947ff-b785-4525-a5f5-3653fc91c99f\") " pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:17.461769 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.461742    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks6tk\" (UniqueName: \"kubernetes.io/projected/ac5947ff-b785-4525-a5f5-3653fc91c99f-kube-api-access-ks6tk\") pod \"payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq\" (UID: \"ac5947ff-b785-4525-a5f5-3653fc91c99f\") " pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:17.538211 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.538175    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:17.816171 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:17.815920    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"]
Sep 15 12:39:17.828848 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:39:17.828814    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5947ff_b785_4525_a5f5_3653fc91c99f.slice/crio-00a09ecc5c2a16ceafa1c0418b2efcddad321db9817c92f2bf171fd53ad378e7 WatchSource:0}: Error finding container 00a09ecc5c2a16ceafa1c0418b2efcddad321db9817c92f2bf171fd53ad378e7: Status 404 returned error can't find the container with id 00a09ecc5c2a16ceafa1c0418b2efcddad321db9817c92f2bf171fd53ad378e7
Sep 15 12:39:18.661072 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:18.661043    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:18.661264 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:18.661088    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq" event={"ID":"ac5947ff-b785-4525-a5f5-3653fc91c99f","Type":"ContainerStarted","Data":"20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b"}
Sep 15 12:39:18.661264 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:18.661108    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq" event={"ID":"ac5947ff-b785-4525-a5f5-3653fc91c99f","Type":"ContainerStarted","Data":"00a09ecc5c2a16ceafa1c0418b2efcddad321db9817c92f2bf171fd53ad378e7"}
Sep 15 12:39:18.679160 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:18.679118    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq" podStartSLOduration=1.6791037 podStartE2EDuration="1.6791037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:39:18.676554686 +0000 UTC m=+826.488466130" watchObservedRunningTime="2026-09-15 12:39:18.6791037 +0000 UTC m=+826.491015160"
Sep 15 12:39:25.708939 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:25.708909    2586 generic.go:356] "Generic (PLEG): container finished" podID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerID="1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3" exitCode=0
Sep 15 12:39:25.709346 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:25.708983    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" event={"ID":"bff85a67-fbc8-4f77-acf4-33305db8e41e","Type":"ContainerDied","Data":"1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3"}
Sep 15 12:39:25.709418 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:25.709383    2586 scope.go:117] "RemoveContainer" containerID="1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3"
Sep 15 12:39:25.710579 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:25.710545    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:25.710679 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:25.710633    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:25.710742 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:25.710679    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:26.467646 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:26.467607    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:39:26.721104 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:26.721001    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" event={"ID":"bff85a67-fbc8-4f77-acf4-33305db8e41e","Type":"ContainerStarted","Data":"ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99"}
Sep 15 12:39:27.111789 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:27.111687    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:39:27.113262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:27.113233    2586 patch_prober.go:28] interesting pod/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 15 12:39:27.113470 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:27.113447    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 15 12:39:28.111584 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:28.111537    2586 patch_prober.go:28] interesting pod/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 15 12:39:28.112141 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:28.111608    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 15 12:39:29.111033 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:29.110997    2586 patch_prober.go:28] interesting pod/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 15 12:39:29.111299 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:29.111272    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 15 12:39:29.668428 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:29.668399    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:30.112189 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:30.111702    2586 patch_prober.go:28] interesting pod/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 15 12:39:30.112189 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:30.111797    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 15 12:39:31.111602 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:31.111137    2586 patch_prober.go:28] interesting pod/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 15 12:39:31.111602 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:31.111213    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 15 12:39:32.123710 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:32.123678    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:39:32.124806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:32.124668    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:39:32.125655 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:32.125635    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:39:36.795400 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:36.795364    2586 generic.go:356] "Generic (PLEG): container finished" podID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerID="4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47" exitCode=0
Sep 15 12:39:36.795861 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:36.795435    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" event={"ID":"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe","Type":"ContainerDied","Data":"4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47"}
Sep 15 12:39:36.795947 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:36.795932    2586 scope.go:117] "RemoveContainer" containerID="4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47"
Sep 15 12:39:36.797220 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:36.797190    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:36.797300 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:36.797272    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:36.797344 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:36.797334    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:37.806540 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:37.806499    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" event={"ID":"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe","Type":"ContainerStarted","Data":"5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c"}
Sep 15 12:39:38.299708 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:38.299245    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:38.300719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:38.300684    2586 patch_prober.go:28] interesting pod/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 12:39:38.300876 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:38.300743    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 12:39:39.300017 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:39.299976    2586 patch_prober.go:28] interesting pod/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 12:39:39.301096 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:39.301053    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 12:39:40.299842 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:40.299702    2586 patch_prober.go:28] interesting pod/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 12:39:40.299842 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:40.299772    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 12:39:41.300445 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:41.299997    2586 patch_prober.go:28] interesting pod/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 12:39:41.300445 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:41.300061    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 12:39:42.299390 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:42.299355    2586 patch_prober.go:28] interesting pod/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 15 12:39:42.299668 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:42.299641    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 15 12:39:42.607883 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:42.607770    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:43.303672 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:43.303642    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:43.304891 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:43.304868    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:39:54.912250 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:54.912212    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"]
Sep 15 12:39:54.912844 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:54.912449    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq" podUID="ac5947ff-b785-4525-a5f5-3653fc91c99f" containerName="payload-processing" containerID="cri-o://20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b" gracePeriod=30
Sep 15 12:39:55.194352 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.194323    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:55.282670 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.282642    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"]
Sep 15 12:39:55.282932 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.282909    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy" containerID="cri-o://5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c" gracePeriod=30
Sep 15 12:39:55.342671 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.342639    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ks6tk\" (UniqueName: \"kubernetes.io/projected/ac5947ff-b785-4525-a5f5-3653fc91c99f-kube-api-access-ks6tk\") pod \"ac5947ff-b785-4525-a5f5-3653fc91c99f\" (UID: \"ac5947ff-b785-4525-a5f5-3653fc91c99f\") "
Sep 15 12:39:55.342853 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.342719    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac5947ff-b785-4525-a5f5-3653fc91c99f-plugins-config-volume\") pod \"ac5947ff-b785-4525-a5f5-3653fc91c99f\" (UID: \"ac5947ff-b785-4525-a5f5-3653fc91c99f\") "
Sep 15 12:39:55.343048 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.343022    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac5947ff-b785-4525-a5f5-3653fc91c99f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac5947ff-b785-4525-a5f5-3653fc91c99f" (UID: "ac5947ff-b785-4525-a5f5-3653fc91c99f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:39:55.344552 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.344527    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac5947ff-b785-4525-a5f5-3653fc91c99f-kube-api-access-ks6tk" (OuterVolumeSpecName: "kube-api-access-ks6tk") pod "ac5947ff-b785-4525-a5f5-3653fc91c99f" (UID: "ac5947ff-b785-4525-a5f5-3653fc91c99f"). InnerVolumeSpecName "kube-api-access-ks6tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:39:55.444454 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.444368    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks6tk\" (UniqueName: \"kubernetes.io/projected/ac5947ff-b785-4525-a5f5-3653fc91c99f-kube-api-access-ks6tk\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:55.444454 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.444396    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac5947ff-b785-4525-a5f5-3653fc91c99f-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:39:55.919882 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.919849    2586 generic.go:356] "Generic (PLEG): container finished" podID="ac5947ff-b785-4525-a5f5-3653fc91c99f" containerID="20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b" exitCode=0
Sep 15 12:39:55.920284 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.919894    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"
Sep 15 12:39:55.920284 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.919939    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq" event={"ID":"ac5947ff-b785-4525-a5f5-3653fc91c99f","Type":"ContainerDied","Data":"20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b"}
Sep 15 12:39:55.920284 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.919986    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq" event={"ID":"ac5947ff-b785-4525-a5f5-3653fc91c99f","Type":"ContainerDied","Data":"00a09ecc5c2a16ceafa1c0418b2efcddad321db9817c92f2bf171fd53ad378e7"}
Sep 15 12:39:55.920284 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.920007    2586 scope.go:117] "RemoveContainer" containerID="20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b"
Sep 15 12:39:55.929650 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.929631    2586 scope.go:117] "RemoveContainer" containerID="20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b"
Sep 15 12:39:55.929954 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:39:55.929928    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b\": container with ID starting with 20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b not found: ID does not exist" containerID="20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b"
Sep 15 12:39:55.930038 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.929964    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b"} err="failed to get container status \"20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b\": rpc error: code = NotFound desc = could not find container \"20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b\": container with ID starting with 20b3665d0c1acffde6a561e5b81943121d56bfead5e007266ccbd54ca24f786b not found: ID does not exist"
Sep 15 12:39:55.942691 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.942670    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"]
Sep 15 12:39:55.949108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:55.949086    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ecae662b-6896d8c7d6-47fmq"]
Sep 15 12:39:56.044243 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.044199    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"]
Sep 15 12:39:56.045061 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.045039    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac5947ff-b785-4525-a5f5-3653fc91c99f" containerName="payload-processing"
Sep 15 12:39:56.045061 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.045059    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac5947ff-b785-4525-a5f5-3653fc91c99f" containerName="payload-processing"
Sep 15 12:39:56.045264 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.045123    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac5947ff-b785-4525-a5f5-3653fc91c99f" containerName="payload-processing"
Sep 15 12:39:56.047629 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.047606    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.049984 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.049964    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-81247cf5-openshift-default-dockercfg-xrx8v\""
Sep 15 12:39:56.061059 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.061037    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"]
Sep 15 12:39:56.152955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.152405    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-envoy\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.152955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.152454    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-podinfo\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.152955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.152512    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-data\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.152955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.152545    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-credential-socket\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.152955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.152581    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk5w2\" (UniqueName: \"kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-kube-api-access-hk5w2\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.152955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.152652    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-token\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.152955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.152691    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-certs\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.152955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.152752    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-socket\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.152955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.152842    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb49573-608c-481f-b662-3e0fd766dfaa-istiod-ca-cert\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254246 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254211    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-credential-socket\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254425 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254253    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk5w2\" (UniqueName: \"kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-kube-api-access-hk5w2\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254425 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254304    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-token\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254425 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254328    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-certs\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254425 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254371    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-socket\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254425 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254410    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb49573-608c-481f-b662-3e0fd766dfaa-istiod-ca-cert\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254443    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-envoy\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254457    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-podinfo\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254506    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-data\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254714    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-certs\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254861    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-credential-socket\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254988 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254896    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-socket\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.254988 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.254906    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-data\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.255141 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.255122    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb49573-608c-481f-b662-3e0fd766dfaa-istiod-ca-cert\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.256750 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.256728    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-envoy\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.256901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.256886    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-podinfo\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.263031 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.263008    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-token\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.263214 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.263182    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk5w2\" (UniqueName: \"kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-kube-api-access-hk5w2\") pod \"e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") " pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.358996 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.358961    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:56.633734 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.633700    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"]
Sep 15 12:39:56.634882 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:39:56.634857    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb49573_608c_481f_b662_3e0fd766dfaa.slice/crio-584fbc81aa1b6e188bb54966ee9cfdf61e98c2aff35dd26f8b4506aab131f5be WatchSource:0}: Error finding container 584fbc81aa1b6e188bb54966ee9cfdf61e98c2aff35dd26f8b4506aab131f5be: Status 404 returned error can't find the container with id 584fbc81aa1b6e188bb54966ee9cfdf61e98c2aff35dd26f8b4506aab131f5be
Sep 15 12:39:56.637184 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.637149    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:56.637281 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.637212    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:56.637281 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.637237    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:39:56.653518 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.653493    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac5947ff-b785-4525-a5f5-3653fc91c99f" path="/var/lib/kubelet/pods/ac5947ff-b785-4525-a5f5-3653fc91c99f/volumes"
Sep 15 12:39:56.926816 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.926714    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" event={"ID":"bcb49573-608c-481f-b662-3e0fd766dfaa","Type":"ContainerStarted","Data":"2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9"}
Sep 15 12:39:56.926816 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.926804    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" event={"ID":"bcb49573-608c-481f-b662-3e0fd766dfaa","Type":"ContainerStarted","Data":"584fbc81aa1b6e188bb54966ee9cfdf61e98c2aff35dd26f8b4506aab131f5be"}
Sep 15 12:39:56.945799 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:56.945689    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" podStartSLOduration=0.945668688 podStartE2EDuration="945.668688ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:39:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:39:56.945431532 +0000 UTC m=+864.757342976" watchObservedRunningTime="2026-09-15 12:39:56.945668688 +0000 UTC m=+864.757580132"
Sep 15 12:39:57.359162 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:57.359132    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:57.364423 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:57.364400    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:57.933578 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:57.933541    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:57.934497 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:57.934473    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:39:58.306321 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:58.306279    2586 patch_prober.go:28] interesting pod/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 12:39:58.306481 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:39:58.306349    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 12:40:00.531911 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.531886    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:40:00.602131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602048    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwx54\" (UniqueName: \"kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-kube-api-access-hwx54\") pod \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") "
Sep 15 12:40:00.602131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602093    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-socket\") pod \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") "
Sep 15 12:40:00.602348 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602145    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-token\") pod \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") "
Sep 15 12:40:00.602348 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602187    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-data\") pod \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") "
Sep 15 12:40:00.602348 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602217    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-envoy\") pod \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") "
Sep 15 12:40:00.602348 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602250    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-credential-socket\") pod \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") "
Sep 15 12:40:00.602348 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602283    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-podinfo\") pod \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") "
Sep 15 12:40:00.602348 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602335    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istiod-ca-cert\") pod \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") "
Sep 15 12:40:00.602637 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602420    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-certs\") pod \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\" (UID: \"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe\") "
Sep 15 12:40:00.602893 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602849    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" (UID: "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:40:00.602893 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602872    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" (UID: "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:40:00.603101 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.602892    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" (UID: "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:40:00.603101 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.603057    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" (UID: "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:40:00.603358 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.603336    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" (UID: "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:40:00.605138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.605109    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" (UID: "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:40:00.605138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.605106    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" (UID: "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:40:00.605300 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.605191    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" (UID: "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:40:00.605300 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.605241    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-kube-api-access-hwx54" (OuterVolumeSpecName: "kube-api-access-hwx54") pod "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" (UID: "8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe"). InnerVolumeSpecName "kube-api-access-hwx54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:40:00.703837 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.703803    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwx54\" (UniqueName: \"kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-kube-api-access-hwx54\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:00.703837 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.703837    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:00.704014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.703851    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:00.704014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.703863    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:00.704014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.703873    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:00.704014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.703883    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:00.704014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.703897    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:00.704014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.703909    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:00.704014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.703920    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:00.949935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.949852    2586 generic.go:356] "Generic (PLEG): container finished" podID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerID="5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c" exitCode=0
Sep 15 12:40:00.949935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.949907    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"
Sep 15 12:40:00.950114 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.949940    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" event={"ID":"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe","Type":"ContainerDied","Data":"5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c"}
Sep 15 12:40:00.950114 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.950000    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg" event={"ID":"8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe","Type":"ContainerDied","Data":"e9b8e7466ab98f8824076dcc30be79bf8248c07b38249e66913631621b185e67"}
Sep 15 12:40:00.950114 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.950032    2586 scope.go:117] "RemoveContainer" containerID="5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c"
Sep 15 12:40:00.958551 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.958532    2586 scope.go:117] "RemoveContainer" containerID="4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47"
Sep 15 12:40:00.967154 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.967128    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"]
Sep 15 12:40:00.967403 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.967390    2586 scope.go:117] "RemoveContainer" containerID="5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c"
Sep 15 12:40:00.967645 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:40:00.967626    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c\": container with ID starting with 5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c not found: ID does not exist" containerID="5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c"
Sep 15 12:40:00.967708 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.967651    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c"} err="failed to get container status \"5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c\": rpc error: code = NotFound desc = could not find container \"5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c\": container with ID starting with 5120530fe2dd3435aab773f07c420c2f9d6c515e083a1150e55ac46d9e90bf8c not found: ID does not exist"
Sep 15 12:40:00.967708 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.967664    2586 scope.go:117] "RemoveContainer" containerID="4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47"
Sep 15 12:40:00.967909 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:40:00.967893    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47\": container with ID starting with 4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47 not found: ID does not exist" containerID="4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47"
Sep 15 12:40:00.967955 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.967912    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47"} err="failed to get container status \"4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47\": rpc error: code = NotFound desc = could not find container \"4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47\": container with ID starting with 4af8d1b58addeb516ebb1daa4c051cace8c5462b9496e69df5412f3a28c1fa47 not found: ID does not exist"
Sep 15 12:40:00.975499 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.975477    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ecae662b-gw-openshift-default-6f95964dcb-mktbg"]
Sep 15 12:40:00.979260 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.979240    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"]
Sep 15 12:40:00.979661 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.979650    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy"
Sep 15 12:40:00.979706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.979663    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy"
Sep 15 12:40:00.979706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.979672    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy"
Sep 15 12:40:00.979706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.979678    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy"
Sep 15 12:40:00.979822 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.979735    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy"
Sep 15 12:40:00.982961 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.982944    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:00.985099 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.985083    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-81247cf5\""
Sep 15 12:40:00.985176 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.985102    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-81247cf5-dockercfg-qbhgt\""
Sep 15 12:40:00.991628 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:00.991609    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"]
Sep 15 12:40:01.007325 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.006999    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6d5f9a2-cf73-455f-997d-d09f114c9e24-plugins-config-volume\") pod \"payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5\" (UID: \"f6d5f9a2-cf73-455f-997d-d09f114c9e24\") " pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:01.007325 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.007086    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmk7f\" (UniqueName: \"kubernetes.io/projected/f6d5f9a2-cf73-455f-997d-d09f114c9e24-kube-api-access-tmk7f\") pod \"payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5\" (UID: \"f6d5f9a2-cf73-455f-997d-d09f114c9e24\") " pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:01.009008 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.008977    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"]
Sep 15 12:40:01.009453 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.009436    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" containerName="istio-proxy"
Sep 15 12:40:01.014352 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.014249    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:01.022746 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.022708    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"]
Sep 15 12:40:01.108317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.108280    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef53671-206f-48c0-8acd-da59e59b32bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd\" (UID: \"5ef53671-206f-48c0-8acd-da59e59b32bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:01.108512 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.108358    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6d5f9a2-cf73-455f-997d-d09f114c9e24-plugins-config-volume\") pod \"payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5\" (UID: \"f6d5f9a2-cf73-455f-997d-d09f114c9e24\") " pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:01.108512 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.108423    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjbrc\" (UniqueName: \"kubernetes.io/projected/5ef53671-206f-48c0-8acd-da59e59b32bf-kube-api-access-xjbrc\") pod \"payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd\" (UID: \"5ef53671-206f-48c0-8acd-da59e59b32bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:01.108512 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.108475    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmk7f\" (UniqueName: \"kubernetes.io/projected/f6d5f9a2-cf73-455f-997d-d09f114c9e24-kube-api-access-tmk7f\") pod \"payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5\" (UID: \"f6d5f9a2-cf73-455f-997d-d09f114c9e24\") " pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:01.109083 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.109060    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6d5f9a2-cf73-455f-997d-d09f114c9e24-plugins-config-volume\") pod \"payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5\" (UID: \"f6d5f9a2-cf73-455f-997d-d09f114c9e24\") " pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:01.116264 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.116243    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmk7f\" (UniqueName: \"kubernetes.io/projected/f6d5f9a2-cf73-455f-997d-d09f114c9e24-kube-api-access-tmk7f\") pod \"payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5\" (UID: \"f6d5f9a2-cf73-455f-997d-d09f114c9e24\") " pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:01.209426 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.209344    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef53671-206f-48c0-8acd-da59e59b32bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd\" (UID: \"5ef53671-206f-48c0-8acd-da59e59b32bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:01.209426 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.209404    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjbrc\" (UniqueName: \"kubernetes.io/projected/5ef53671-206f-48c0-8acd-da59e59b32bf-kube-api-access-xjbrc\") pod \"payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd\" (UID: \"5ef53671-206f-48c0-8acd-da59e59b32bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:01.210141 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.210121    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef53671-206f-48c0-8acd-da59e59b32bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd\" (UID: \"5ef53671-206f-48c0-8acd-da59e59b32bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:01.217454 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.217421    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjbrc\" (UniqueName: \"kubernetes.io/projected/5ef53671-206f-48c0-8acd-da59e59b32bf-kube-api-access-xjbrc\") pod \"payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd\" (UID: \"5ef53671-206f-48c0-8acd-da59e59b32bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:01.294146 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.294115    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:01.328766 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.328314    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:01.584891 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.584867    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"]
Sep 15 12:40:01.586231 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:40:01.586208    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d5f9a2_cf73_455f_997d_d09f114c9e24.slice/crio-9815a0f53c1fc06bb9004cd1b1b3dac5e0976dfc3c3c46087629b93a05cbe5de WatchSource:0}: Error finding container 9815a0f53c1fc06bb9004cd1b1b3dac5e0976dfc3c3c46087629b93a05cbe5de: Status 404 returned error can't find the container with id 9815a0f53c1fc06bb9004cd1b1b3dac5e0976dfc3c3c46087629b93a05cbe5de
Sep 15 12:40:01.613424 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.613392    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"]
Sep 15 12:40:01.618747 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:40:01.618721    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef53671_206f_48c0_8acd_da59e59b32bf.slice/crio-5e805cda551e953df5a88a56b20993743516612b01321ac1d314bd10b7e66a7c WatchSource:0}: Error finding container 5e805cda551e953df5a88a56b20993743516612b01321ac1d314bd10b7e66a7c: Status 404 returned error can't find the container with id 5e805cda551e953df5a88a56b20993743516612b01321ac1d314bd10b7e66a7c
Sep 15 12:40:01.959102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.959004    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd" event={"ID":"5ef53671-206f-48c0-8acd-da59e59b32bf","Type":"ContainerStarted","Data":"e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775"}
Sep 15 12:40:01.959102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.959071    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:01.959102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.959098    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd" event={"ID":"5ef53671-206f-48c0-8acd-da59e59b32bf","Type":"ContainerStarted","Data":"5e805cda551e953df5a88a56b20993743516612b01321ac1d314bd10b7e66a7c"}
Sep 15 12:40:01.960637 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.960608    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5" event={"ID":"f6d5f9a2-cf73-455f-997d-d09f114c9e24","Type":"ContainerStarted","Data":"adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29"}
Sep 15 12:40:01.960752 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.960653    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:01.960752 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.960670    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5" event={"ID":"f6d5f9a2-cf73-455f-997d-d09f114c9e24","Type":"ContainerStarted","Data":"9815a0f53c1fc06bb9004cd1b1b3dac5e0976dfc3c3c46087629b93a05cbe5de"}
Sep 15 12:40:01.974099 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.973860    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd" podStartSLOduration=1.9738399800000002 podStartE2EDuration="1.97383998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:40:01.973725257 +0000 UTC m=+869.785636700" watchObservedRunningTime="2026-09-15 12:40:01.97383998 +0000 UTC m=+869.785751424"
Sep 15 12:40:01.990143 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:01.990096    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5" podStartSLOduration=1.990085125 podStartE2EDuration="1.990085125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:40:01.988020031 +0000 UTC m=+869.799931486" watchObservedRunningTime="2026-09-15 12:40:01.990085125 +0000 UTC m=+869.801996568"
Sep 15 12:40:02.655066 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:02.655033    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe" path="/var/lib/kubelet/pods/8f8c7cfe-0ba1-4045-936f-1f78bd1e8afe/volumes"
Sep 15 12:40:12.967951 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:12.967921    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:12.968407 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:12.968093    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:25.098878 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:25.098795    2586 generic.go:356] "Generic (PLEG): container finished" podID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerID="2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9" exitCode=0
Sep 15 12:40:25.098878 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:25.098816    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" event={"ID":"bcb49573-608c-481f-b662-3e0fd766dfaa","Type":"ContainerDied","Data":"2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9"}
Sep 15 12:40:25.099438 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:25.099420    2586 scope.go:117] "RemoveContainer" containerID="2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9"
Sep 15 12:40:25.101056 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:25.101022    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:40:25.101152 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:25.101108    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:40:25.101195 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:25.101153    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:40:26.110642 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:26.110600    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" event={"ID":"bcb49573-608c-481f-b662-3e0fd766dfaa","Type":"ContainerStarted","Data":"342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1"}
Sep 15 12:40:26.359856 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:26.359823    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:40:26.361092 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:26.361034    2586 patch_prober.go:28] interesting pod/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 15 12:40:26.361092 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:26.361082    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 15 12:40:27.359897 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:27.359860    2586 patch_prober.go:28] interesting pod/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 15 12:40:27.360262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:27.359912    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 15 12:40:27.933574 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:27.933543    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:40:28.359701 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:28.359666    2586 patch_prober.go:28] interesting pod/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 15 12:40:28.359929 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:28.359716    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 15 12:40:29.360094 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:29.360055    2586 patch_prober.go:28] interesting pod/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 15 12:40:29.360476 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:29.360112    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 15 12:40:30.359880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:30.359843    2586 patch_prober.go:28] interesting pod/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 15 12:40:30.360042 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:30.359900    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 15 12:40:30.575187 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:30.575141    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"]
Sep 15 12:40:30.576499 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:30.575941    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5" podUID="f6d5f9a2-cf73-455f-997d-d09f114c9e24" containerName="payload-processing" containerID="cri-o://adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29" gracePeriod=30
Sep 15 12:40:30.584938 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:30.584913    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"]
Sep 15 12:40:30.585388 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:30.585342    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd" podUID="5ef53671-206f-48c0-8acd-da59e59b32bf" containerName="payload-pre-processing" containerID="cri-o://e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775" gracePeriod=30
Sep 15 12:40:30.901801 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:30.900366    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:30.970839 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:30.970809    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:31.035853 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.035818    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef53671-206f-48c0-8acd-da59e59b32bf-plugins-config-volume\") pod \"5ef53671-206f-48c0-8acd-da59e59b32bf\" (UID: \"5ef53671-206f-48c0-8acd-da59e59b32bf\") "
Sep 15 12:40:31.036057 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.035877    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6d5f9a2-cf73-455f-997d-d09f114c9e24-plugins-config-volume\") pod \"f6d5f9a2-cf73-455f-997d-d09f114c9e24\" (UID: \"f6d5f9a2-cf73-455f-997d-d09f114c9e24\") "
Sep 15 12:40:31.036057 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.035922    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmk7f\" (UniqueName: \"kubernetes.io/projected/f6d5f9a2-cf73-455f-997d-d09f114c9e24-kube-api-access-tmk7f\") pod \"f6d5f9a2-cf73-455f-997d-d09f114c9e24\" (UID: \"f6d5f9a2-cf73-455f-997d-d09f114c9e24\") "
Sep 15 12:40:31.036057 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.035985    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjbrc\" (UniqueName: \"kubernetes.io/projected/5ef53671-206f-48c0-8acd-da59e59b32bf-kube-api-access-xjbrc\") pod \"5ef53671-206f-48c0-8acd-da59e59b32bf\" (UID: \"5ef53671-206f-48c0-8acd-da59e59b32bf\") "
Sep 15 12:40:31.036298 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.036234    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ef53671-206f-48c0-8acd-da59e59b32bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ef53671-206f-48c0-8acd-da59e59b32bf" (UID: "5ef53671-206f-48c0-8acd-da59e59b32bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:40:31.036391 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.036368    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef53671-206f-48c0-8acd-da59e59b32bf-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:31.037040 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.036999    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6d5f9a2-cf73-455f-997d-d09f114c9e24-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6d5f9a2-cf73-455f-997d-d09f114c9e24" (UID: "f6d5f9a2-cf73-455f-997d-d09f114c9e24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:40:31.038580 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.038550    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ef53671-206f-48c0-8acd-da59e59b32bf-kube-api-access-xjbrc" (OuterVolumeSpecName: "kube-api-access-xjbrc") pod "5ef53671-206f-48c0-8acd-da59e59b32bf" (UID: "5ef53671-206f-48c0-8acd-da59e59b32bf"). InnerVolumeSpecName "kube-api-access-xjbrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:40:31.039120 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.039091    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6d5f9a2-cf73-455f-997d-d09f114c9e24-kube-api-access-tmk7f" (OuterVolumeSpecName: "kube-api-access-tmk7f") pod "f6d5f9a2-cf73-455f-997d-d09f114c9e24" (UID: "f6d5f9a2-cf73-455f-997d-d09f114c9e24"). InnerVolumeSpecName "kube-api-access-tmk7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:40:31.137019 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.136985    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6d5f9a2-cf73-455f-997d-d09f114c9e24-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:31.137019 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.137016    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmk7f\" (UniqueName: \"kubernetes.io/projected/f6d5f9a2-cf73-455f-997d-d09f114c9e24-kube-api-access-tmk7f\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:31.137019 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.137026    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjbrc\" (UniqueName: \"kubernetes.io/projected/5ef53671-206f-48c0-8acd-da59e59b32bf-kube-api-access-xjbrc\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:31.143075 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.143047    2586 generic.go:356] "Generic (PLEG): container finished" podID="5ef53671-206f-48c0-8acd-da59e59b32bf" containerID="e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775" exitCode=0
Sep 15 12:40:31.143212 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.143112    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"
Sep 15 12:40:31.143212 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.143141    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd" event={"ID":"5ef53671-206f-48c0-8acd-da59e59b32bf","Type":"ContainerDied","Data":"e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775"}
Sep 15 12:40:31.143212 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.143204    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd" event={"ID":"5ef53671-206f-48c0-8acd-da59e59b32bf","Type":"ContainerDied","Data":"5e805cda551e953df5a88a56b20993743516612b01321ac1d314bd10b7e66a7c"}
Sep 15 12:40:31.143351 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.143239    2586 scope.go:117] "RemoveContainer" containerID="e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775"
Sep 15 12:40:31.144625 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.144602    2586 generic.go:356] "Generic (PLEG): container finished" podID="f6d5f9a2-cf73-455f-997d-d09f114c9e24" containerID="adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29" exitCode=0
Sep 15 12:40:31.144735 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.144648    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"
Sep 15 12:40:31.144735 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.144636    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5" event={"ID":"f6d5f9a2-cf73-455f-997d-d09f114c9e24","Type":"ContainerDied","Data":"adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29"}
Sep 15 12:40:31.144735 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.144728    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5" event={"ID":"f6d5f9a2-cf73-455f-997d-d09f114c9e24","Type":"ContainerDied","Data":"9815a0f53c1fc06bb9004cd1b1b3dac5e0976dfc3c3c46087629b93a05cbe5de"}
Sep 15 12:40:31.153999 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.153981    2586 scope.go:117] "RemoveContainer" containerID="e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775"
Sep 15 12:40:31.154276 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:40:31.154257    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775\": container with ID starting with e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775 not found: ID does not exist" containerID="e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775"
Sep 15 12:40:31.154350 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.154287    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775"} err="failed to get container status \"e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775\": rpc error: code = NotFound desc = could not find container \"e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775\": container with ID starting with e5f92b435ac16bb82d9991b941e7abee57a7168819fd31068c845bd073818775 not found: ID does not exist"
Sep 15 12:40:31.154350 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.154309    2586 scope.go:117] "RemoveContainer" containerID="adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29"
Sep 15 12:40:31.164148 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.164133    2586 scope.go:117] "RemoveContainer" containerID="adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29"
Sep 15 12:40:31.164418 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:40:31.164401    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29\": container with ID starting with adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29 not found: ID does not exist" containerID="adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29"
Sep 15 12:40:31.164466 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.164424    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29"} err="failed to get container status \"adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29\": rpc error: code = NotFound desc = could not find container \"adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29\": container with ID starting with adcae03aacfbaed5fa770fe46d1a7dbbb1bcdc7ad2037c20691b65fa05a91e29 not found: ID does not exist"
Sep 15 12:40:31.171434 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.171412    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"]
Sep 15 12:40:31.177443 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.177422    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-81247cf5-55dbdff4ff-wrvg5"]
Sep 15 12:40:31.188593 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.188574    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"]
Sep 15 12:40:31.191807 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.191773    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-81247cf5-58597bc954-255rd"]
Sep 15 12:40:31.416299 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.416264    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:40:31.420068 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.420041    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:40:31.891097 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:31.891061    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"]
Sep 15 12:40:32.154902 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.154821    2586 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" secret="" err="secret \"e2e-ait-81247cf5-openshift-default-dockercfg-xrx8v\" not found"
Sep 15 12:40:32.580535 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.580509    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"]
Sep 15 12:40:32.581340 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.581314    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6d5f9a2-cf73-455f-997d-d09f114c9e24" containerName="payload-processing"
Sep 15 12:40:32.581340 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.581337    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d5f9a2-cf73-455f-997d-d09f114c9e24" containerName="payload-processing"
Sep 15 12:40:32.581564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.581369    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ef53671-206f-48c0-8acd-da59e59b32bf" containerName="payload-pre-processing"
Sep 15 12:40:32.581564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.581378    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ef53671-206f-48c0-8acd-da59e59b32bf" containerName="payload-pre-processing"
Sep 15 12:40:32.581564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.581444    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ef53671-206f-48c0-8acd-da59e59b32bf" containerName="payload-pre-processing"
Sep 15 12:40:32.581564 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.581461    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6d5f9a2-cf73-455f-997d-d09f114c9e24" containerName="payload-processing"
Sep 15 12:40:32.584139 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.584118    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.586505 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.586487    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-1459b7a2-openshift-default-dockercfg-nhwg6\""
Sep 15 12:40:32.611724 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.611692    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"]
Sep 15 12:40:32.649889 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.649863    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-credential-socket\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.650030 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.649947    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-data\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.650030 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.649993    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-token\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.650030 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.650025    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-socket\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.650138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.650044    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9q2v\" (UniqueName: \"kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-kube-api-access-m9q2v\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.650138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.650061    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-podinfo\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.650138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.650082    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-envoy\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.650247 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.650140    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-certs\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.650247 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.650226    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ecc6aac-33e1-41ad-9768-a65e489f9418-istiod-ca-cert\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.652718 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.652696    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ef53671-206f-48c0-8acd-da59e59b32bf" path="/var/lib/kubelet/pods/5ef53671-206f-48c0-8acd-da59e59b32bf/volumes"
Sep 15 12:40:32.653115 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.653102    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6d5f9a2-cf73-455f-997d-d09f114c9e24" path="/var/lib/kubelet/pods/f6d5f9a2-cf73-455f-997d-d09f114c9e24/volumes"
Sep 15 12:40:32.751138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751114    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-data\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751239 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751168    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-token\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751239 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751195    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-socket\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751239 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751214    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9q2v\" (UniqueName: \"kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-kube-api-access-m9q2v\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751239 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751231    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-podinfo\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751458 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751246    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-envoy\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751458 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751287    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-certs\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751563 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751451    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ecc6aac-33e1-41ad-9768-a65e489f9418-istiod-ca-cert\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751563 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751519    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-data\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751672 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751559    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-socket\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751730 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751671    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-credential-socket\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751816 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751723    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-certs\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.751968 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.751947    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-credential-socket\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.752168 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.752153    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ecc6aac-33e1-41ad-9768-a65e489f9418-istiod-ca-cert\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.754235 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.754211    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-envoy\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.754539 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.754514    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-podinfo\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.759454 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.759430    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9q2v\" (UniqueName: \"kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-kube-api-access-m9q2v\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.759576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.759560    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-token\") pod \"e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") " pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:32.905822 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:32.905744    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:33.163223 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:33.163120    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy" containerID="cri-o://342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1" gracePeriod=30
Sep 15 12:40:33.188743 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:33.188721    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"]
Sep 15 12:40:33.190931 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:40:33.190902    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ecc6aac_33e1_41ad_9768_a65e489f9418.slice/crio-29dddbe9f9bf10210d8384add2d086531ce62ded4116ad4cd859ca98ae39cde8 WatchSource:0}: Error finding container 29dddbe9f9bf10210d8384add2d086531ce62ded4116ad4cd859ca98ae39cde8: Status 404 returned error can't find the container with id 29dddbe9f9bf10210d8384add2d086531ce62ded4116ad4cd859ca98ae39cde8
Sep 15 12:40:33.193246 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:33.193209    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:40:33.193339 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:33.193277    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:40:33.193339 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:33.193311    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:40:34.170480 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:34.170443    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" event={"ID":"4ecc6aac-33e1-41ad-9768-a65e489f9418","Type":"ContainerStarted","Data":"8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89"}
Sep 15 12:40:34.170891 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:34.170498    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" event={"ID":"4ecc6aac-33e1-41ad-9768-a65e489f9418","Type":"ContainerStarted","Data":"29dddbe9f9bf10210d8384add2d086531ce62ded4116ad4cd859ca98ae39cde8"}
Sep 15 12:40:34.190160 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:34.190114    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" podStartSLOduration=2.190099911 podStartE2EDuration="2.190099911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:40:34.188800443 +0000 UTC m=+902.000711880" watchObservedRunningTime="2026-09-15 12:40:34.190099911 +0000 UTC m=+902.002011355"
Sep 15 12:40:34.906936 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:34.906907    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:34.911881 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:34.911858    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:35.175734 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:35.175657    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:35.176584 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:35.176564    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:40:38.421261 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.421235    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:40:38.514561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.514533    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-token\") pod \"bcb49573-608c-481f-b662-3e0fd766dfaa\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") "
Sep 15 12:40:38.514714 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.514631    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-socket\") pod \"bcb49573-608c-481f-b662-3e0fd766dfaa\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") "
Sep 15 12:40:38.514714 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.514664    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb49573-608c-481f-b662-3e0fd766dfaa-istiod-ca-cert\") pod \"bcb49573-608c-481f-b662-3e0fd766dfaa\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") "
Sep 15 12:40:38.514714 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.514681    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-envoy\") pod \"bcb49573-608c-481f-b662-3e0fd766dfaa\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") "
Sep 15 12:40:38.514714 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.514699    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-credential-socket\") pod \"bcb49573-608c-481f-b662-3e0fd766dfaa\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") "
Sep 15 12:40:38.514959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.514715    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-certs\") pod \"bcb49573-608c-481f-b662-3e0fd766dfaa\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") "
Sep 15 12:40:38.514959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.514750    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-podinfo\") pod \"bcb49573-608c-481f-b662-3e0fd766dfaa\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") "
Sep 15 12:40:38.515030 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.514959    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bcb49573-608c-481f-b662-3e0fd766dfaa" (UID: "bcb49573-608c-481f-b662-3e0fd766dfaa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:40:38.515030 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515016    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-data\") pod \"bcb49573-608c-481f-b662-3e0fd766dfaa\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") "
Sep 15 12:40:38.515124 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515052    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk5w2\" (UniqueName: \"kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-kube-api-access-hk5w2\") pod \"bcb49573-608c-481f-b662-3e0fd766dfaa\" (UID: \"bcb49573-608c-481f-b662-3e0fd766dfaa\") "
Sep 15 12:40:38.515124 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515075    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb49573-608c-481f-b662-3e0fd766dfaa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bcb49573-608c-481f-b662-3e0fd766dfaa" (UID: "bcb49573-608c-481f-b662-3e0fd766dfaa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:40:38.515124 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515084    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bcb49573-608c-481f-b662-3e0fd766dfaa" (UID: "bcb49573-608c-481f-b662-3e0fd766dfaa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:40:38.515281 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515208    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bcb49573-608c-481f-b662-3e0fd766dfaa" (UID: "bcb49573-608c-481f-b662-3e0fd766dfaa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:40:38.515387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515351    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-data" (OuterVolumeSpecName: "istio-data") pod "bcb49573-608c-481f-b662-3e0fd766dfaa" (UID: "bcb49573-608c-481f-b662-3e0fd766dfaa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:40:38.515590 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515567    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:38.515590 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515592    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb49573-608c-481f-b662-3e0fd766dfaa-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:38.515766 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515607    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:38.515766 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515620    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:38.515766 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.515633    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:38.517066 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.517045    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-token" (OuterVolumeSpecName: "istio-token") pod "bcb49573-608c-481f-b662-3e0fd766dfaa" (UID: "bcb49573-608c-481f-b662-3e0fd766dfaa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:40:38.517219 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.517191    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bcb49573-608c-481f-b662-3e0fd766dfaa" (UID: "bcb49573-608c-481f-b662-3e0fd766dfaa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:40:38.517599 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.517580    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bcb49573-608c-481f-b662-3e0fd766dfaa" (UID: "bcb49573-608c-481f-b662-3e0fd766dfaa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:40:38.517599 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.517586    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-kube-api-access-hk5w2" (OuterVolumeSpecName: "kube-api-access-hk5w2") pod "bcb49573-608c-481f-b662-3e0fd766dfaa" (UID: "bcb49573-608c-481f-b662-3e0fd766dfaa"). InnerVolumeSpecName "kube-api-access-hk5w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:40:38.617073 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.617031    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk5w2\" (UniqueName: \"kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-kube-api-access-hk5w2\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:38.617073 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.617069    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:38.617283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.617081    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:38.617283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:38.617093    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb49573-608c-481f-b662-3e0fd766dfaa-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:40:39.203415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.203382    2586 generic.go:356] "Generic (PLEG): container finished" podID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerID="342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1" exitCode=0
Sep 15 12:40:39.203621 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.203445    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"
Sep 15 12:40:39.203621 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.203454    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" event={"ID":"bcb49573-608c-481f-b662-3e0fd766dfaa","Type":"ContainerDied","Data":"342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1"}
Sep 15 12:40:39.203621 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.203510    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2" event={"ID":"bcb49573-608c-481f-b662-3e0fd766dfaa","Type":"ContainerDied","Data":"584fbc81aa1b6e188bb54966ee9cfdf61e98c2aff35dd26f8b4506aab131f5be"}
Sep 15 12:40:39.203621 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.203540    2586 scope.go:117] "RemoveContainer" containerID="342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1"
Sep 15 12:40:39.212361 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.212338    2586 scope.go:117] "RemoveContainer" containerID="2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9"
Sep 15 12:40:39.220124 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.220108    2586 scope.go:117] "RemoveContainer" containerID="342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1"
Sep 15 12:40:39.220423 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:40:39.220405    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1\": container with ID starting with 342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1 not found: ID does not exist" containerID="342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1"
Sep 15 12:40:39.220489 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.220431    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1"} err="failed to get container status \"342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1\": rpc error: code = NotFound desc = could not find container \"342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1\": container with ID starting with 342f59f3191f59608eeb31ef3ed7fa930fa2fbfcc7471d4bbb15e83a321a81b1 not found: ID does not exist"
Sep 15 12:40:39.220489 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.220450    2586 scope.go:117] "RemoveContainer" containerID="2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9"
Sep 15 12:40:39.220683 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:40:39.220667    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9\": container with ID starting with 2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9 not found: ID does not exist" containerID="2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9"
Sep 15 12:40:39.220734 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.220692    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9"} err="failed to get container status \"2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9\": rpc error: code = NotFound desc = could not find container \"2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9\": container with ID starting with 2da412a70f93ca2e694ae753188181c56578d1cef49fd76b833607ce087ca0c9 not found: ID does not exist"
Sep 15 12:40:39.227177 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.227155    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"]
Sep 15 12:40:39.234166 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:39.234146    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-81247cf5-openshift-default-6bcf967bd-kqvd2"]
Sep 15 12:40:40.363382 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.363344    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"]
Sep 15 12:40:40.363871 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.363855    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy"
Sep 15 12:40:40.363871 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.363873    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy"
Sep 15 12:40:40.363956 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.363884    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy"
Sep 15 12:40:40.363956 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.363889    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy"
Sep 15 12:40:40.364021 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.363990    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy"
Sep 15 12:40:40.364021 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.364000    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" containerName="istio-proxy"
Sep 15 12:40:40.367282 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.367267    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:40:40.369579 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.369560    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-1459b7a2\""
Sep 15 12:40:40.369681 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.369602    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-1459b7a2-dockercfg-w7q2q\""
Sep 15 12:40:40.377436 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.377414    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"]
Sep 15 12:40:40.400857 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.400831    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"]
Sep 15 12:40:40.405003 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.404983    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:40:40.409874 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.409854    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"]
Sep 15 12:40:40.438576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.438551    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wvx2\" (UniqueName: \"kubernetes.io/projected/1a198470-3d87-4562-8839-71a990a5031e-kube-api-access-5wvx2\") pod \"payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h\" (UID: \"1a198470-3d87-4562-8839-71a990a5031e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:40:40.438706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.438625    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a198470-3d87-4562-8839-71a990a5031e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h\" (UID: \"1a198470-3d87-4562-8839-71a990a5031e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:40:40.438706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.438655    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr7lx\" (UniqueName: \"kubernetes.io/projected/bedfbdac-c04a-4a41-a1cc-619a5475b094-kube-api-access-jr7lx\") pod \"payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8\" (UID: \"bedfbdac-c04a-4a41-a1cc-619a5475b094\") " pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:40:40.438828 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.438805    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bedfbdac-c04a-4a41-a1cc-619a5475b094-plugins-config-volume\") pod \"payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8\" (UID: \"bedfbdac-c04a-4a41-a1cc-619a5475b094\") " pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:40:40.540233 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.540198    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wvx2\" (UniqueName: \"kubernetes.io/projected/1a198470-3d87-4562-8839-71a990a5031e-kube-api-access-5wvx2\") pod \"payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h\" (UID: \"1a198470-3d87-4562-8839-71a990a5031e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:40:40.540422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.540260    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a198470-3d87-4562-8839-71a990a5031e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h\" (UID: \"1a198470-3d87-4562-8839-71a990a5031e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:40:40.540422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.540279    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr7lx\" (UniqueName: \"kubernetes.io/projected/bedfbdac-c04a-4a41-a1cc-619a5475b094-kube-api-access-jr7lx\") pod \"payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8\" (UID: \"bedfbdac-c04a-4a41-a1cc-619a5475b094\") " pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:40:40.540422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.540327    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bedfbdac-c04a-4a41-a1cc-619a5475b094-plugins-config-volume\") pod \"payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8\" (UID: \"bedfbdac-c04a-4a41-a1cc-619a5475b094\") " pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:40:40.541032 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.541011    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bedfbdac-c04a-4a41-a1cc-619a5475b094-plugins-config-volume\") pod \"payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8\" (UID: \"bedfbdac-c04a-4a41-a1cc-619a5475b094\") " pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:40:40.541111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.541089    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a198470-3d87-4562-8839-71a990a5031e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h\" (UID: \"1a198470-3d87-4562-8839-71a990a5031e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:40:40.548234 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.548209    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr7lx\" (UniqueName: \"kubernetes.io/projected/bedfbdac-c04a-4a41-a1cc-619a5475b094-kube-api-access-jr7lx\") pod \"payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8\" (UID: \"bedfbdac-c04a-4a41-a1cc-619a5475b094\") " pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:40:40.548336 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.548248    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wvx2\" (UniqueName: \"kubernetes.io/projected/1a198470-3d87-4562-8839-71a990a5031e-kube-api-access-5wvx2\") pod \"payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h\" (UID: \"1a198470-3d87-4562-8839-71a990a5031e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:40:40.653923 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.653852    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcb49573-608c-481f-b662-3e0fd766dfaa" path="/var/lib/kubelet/pods/bcb49573-608c-481f-b662-3e0fd766dfaa/volumes"
Sep 15 12:40:40.678738 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.678708    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:40:40.718244 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.718204    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:40:40.950840 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:40.950818    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"]
Sep 15 12:40:40.952520 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:40:40.952495    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a198470_3d87_4562_8839_71a990a5031e.slice/crio-e59169b945a2769e89dfba4026464f5ea01a9bce6809da631ffbc65533215a3d WatchSource:0}: Error finding container e59169b945a2769e89dfba4026464f5ea01a9bce6809da631ffbc65533215a3d: Status 404 returned error can't find the container with id e59169b945a2769e89dfba4026464f5ea01a9bce6809da631ffbc65533215a3d
Sep 15 12:40:41.000694 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:41.000662    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"]
Sep 15 12:40:41.001982 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:40:41.001957    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbedfbdac_c04a_4a41_a1cc_619a5475b094.slice/crio-fe28f510a390138cace2b950dd168d4a9a982b5b9759a1ea5e8a051dd4fb6e5a WatchSource:0}: Error finding container fe28f510a390138cace2b950dd168d4a9a982b5b9759a1ea5e8a051dd4fb6e5a: Status 404 returned error can't find the container with id fe28f510a390138cace2b950dd168d4a9a982b5b9759a1ea5e8a051dd4fb6e5a
Sep 15 12:40:41.219528 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:41.219443    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8" event={"ID":"bedfbdac-c04a-4a41-a1cc-619a5475b094","Type":"ContainerStarted","Data":"f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40"}
Sep 15 12:40:41.219528 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:41.219505    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8" event={"ID":"bedfbdac-c04a-4a41-a1cc-619a5475b094","Type":"ContainerStarted","Data":"fe28f510a390138cace2b950dd168d4a9a982b5b9759a1ea5e8a051dd4fb6e5a"}
Sep 15 12:40:41.219766 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:41.219550    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:40:41.221419 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:41.221398    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h" event={"ID":"1a198470-3d87-4562-8839-71a990a5031e","Type":"ContainerStarted","Data":"7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55"}
Sep 15 12:40:41.221548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:41.221439    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h" event={"ID":"1a198470-3d87-4562-8839-71a990a5031e","Type":"ContainerStarted","Data":"e59169b945a2769e89dfba4026464f5ea01a9bce6809da631ffbc65533215a3d"}
Sep 15 12:40:41.221548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:41.221479    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:40:41.236448 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:41.236402    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8" podStartSLOduration=1.236361282 podStartE2EDuration="1.236361282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:40:41.234891048 +0000 UTC m=+909.046802489" watchObservedRunningTime="2026-09-15 12:40:41.236361282 +0000 UTC m=+909.048272724"
Sep 15 12:40:41.251712 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:41.251670    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h" podStartSLOduration=1.251658829 podStartE2EDuration="1.251658829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:40:41.249507667 +0000 UTC m=+909.061419123" watchObservedRunningTime="2026-09-15 12:40:41.251658829 +0000 UTC m=+909.063570271"
Sep 15 12:40:52.229750 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:52.229721    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:40:52.230256 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:40:52.229849    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:41:03.355984 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:03.355950    2586 generic.go:356] "Generic (PLEG): container finished" podID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerID="8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89" exitCode=0
Sep 15 12:41:03.356386 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:03.356023    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" event={"ID":"4ecc6aac-33e1-41ad-9768-a65e489f9418","Type":"ContainerDied","Data":"8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89"}
Sep 15 12:41:03.356434 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:03.356424    2586 scope.go:117] "RemoveContainer" containerID="8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89"
Sep 15 12:41:03.357522 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:03.357496    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:41:03.357577 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:03.357561    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:41:03.357612 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:03.357591    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:41:04.363863 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:04.363829    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" event={"ID":"4ecc6aac-33e1-41ad-9768-a65e489f9418","Type":"ContainerStarted","Data":"80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f"}
Sep 15 12:41:04.908009 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:04.907966    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:41:04.908223 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:04.908136    2586 patch_prober.go:28] interesting pod/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 15 12:41:04.908223 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:04.908183    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 15 12:41:05.176095 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:05.176010    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:41:05.907222 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:05.906732    2586 patch_prober.go:28] interesting pod/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 15 12:41:05.907222 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:05.906825    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 15 12:41:06.906472 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:06.906430    2586 patch_prober.go:28] interesting pod/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 15 12:41:06.906663 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:06.906503    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 15 12:41:07.687205 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:07.687171    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"]
Sep 15 12:41:07.687825 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:07.687418    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8" podUID="bedfbdac-c04a-4a41-a1cc-619a5475b094" containerName="payload-processing" containerID="cri-o://f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40" gracePeriod=30
Sep 15 12:41:07.692155 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:07.692031    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"]
Sep 15 12:41:07.692745 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:07.692211    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h" podUID="1a198470-3d87-4562-8839-71a990a5031e" containerName="payload-pre-processing" containerID="cri-o://7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55" gracePeriod=30
Sep 15 12:41:07.919231 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:07.919183    2586 patch_prober.go:28] interesting pod/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 15 12:41:07.919413 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:07.919255    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 15 12:41:08.034811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.025310    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"]
Sep 15 12:41:08.034811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.025544    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy" containerID="cri-o://80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f" gracePeriod=30
Sep 15 12:41:08.078307 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.078274    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:41:08.081467 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.081441    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:41:08.147356 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.141361    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr7lx\" (UniqueName: \"kubernetes.io/projected/bedfbdac-c04a-4a41-a1cc-619a5475b094-kube-api-access-jr7lx\") pod \"bedfbdac-c04a-4a41-a1cc-619a5475b094\" (UID: \"bedfbdac-c04a-4a41-a1cc-619a5475b094\") "
Sep 15 12:41:08.147356 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.141410    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a198470-3d87-4562-8839-71a990a5031e-plugins-config-volume\") pod \"1a198470-3d87-4562-8839-71a990a5031e\" (UID: \"1a198470-3d87-4562-8839-71a990a5031e\") "
Sep 15 12:41:08.147356 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.141447    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wvx2\" (UniqueName: \"kubernetes.io/projected/1a198470-3d87-4562-8839-71a990a5031e-kube-api-access-5wvx2\") pod \"1a198470-3d87-4562-8839-71a990a5031e\" (UID: \"1a198470-3d87-4562-8839-71a990a5031e\") "
Sep 15 12:41:08.147356 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.141488    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bedfbdac-c04a-4a41-a1cc-619a5475b094-plugins-config-volume\") pod \"bedfbdac-c04a-4a41-a1cc-619a5475b094\" (UID: \"bedfbdac-c04a-4a41-a1cc-619a5475b094\") "
Sep 15 12:41:08.147356 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.142611    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a198470-3d87-4562-8839-71a990a5031e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a198470-3d87-4562-8839-71a990a5031e" (UID: "1a198470-3d87-4562-8839-71a990a5031e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:41:08.147356 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.143053    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bedfbdac-c04a-4a41-a1cc-619a5475b094-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bedfbdac-c04a-4a41-a1cc-619a5475b094" (UID: "bedfbdac-c04a-4a41-a1cc-619a5475b094"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:41:08.153811 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.152201    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a198470-3d87-4562-8839-71a990a5031e-kube-api-access-5wvx2" (OuterVolumeSpecName: "kube-api-access-5wvx2") pod "1a198470-3d87-4562-8839-71a990a5031e" (UID: "1a198470-3d87-4562-8839-71a990a5031e"). InnerVolumeSpecName "kube-api-access-5wvx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:41:08.156838 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.154993    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bedfbdac-c04a-4a41-a1cc-619a5475b094-kube-api-access-jr7lx" (OuterVolumeSpecName: "kube-api-access-jr7lx") pod "bedfbdac-c04a-4a41-a1cc-619a5475b094" (UID: "bedfbdac-c04a-4a41-a1cc-619a5475b094"). InnerVolumeSpecName "kube-api-access-jr7lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:41:08.242364 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.242335    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr7lx\" (UniqueName: \"kubernetes.io/projected/bedfbdac-c04a-4a41-a1cc-619a5475b094-kube-api-access-jr7lx\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:08.242364 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.242363    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a198470-3d87-4562-8839-71a990a5031e-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:08.242556 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.242373    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wvx2\" (UniqueName: \"kubernetes.io/projected/1a198470-3d87-4562-8839-71a990a5031e-kube-api-access-5wvx2\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:08.242556 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.242382    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bedfbdac-c04a-4a41-a1cc-619a5475b094-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:08.392001 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.391962    2586 generic.go:356] "Generic (PLEG): container finished" podID="bedfbdac-c04a-4a41-a1cc-619a5475b094" containerID="f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40" exitCode=0
Sep 15 12:41:08.392183 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.392008    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"
Sep 15 12:41:08.392183 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.392046    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8" event={"ID":"bedfbdac-c04a-4a41-a1cc-619a5475b094","Type":"ContainerDied","Data":"f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40"}
Sep 15 12:41:08.392183 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.392112    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8" event={"ID":"bedfbdac-c04a-4a41-a1cc-619a5475b094","Type":"ContainerDied","Data":"fe28f510a390138cace2b950dd168d4a9a982b5b9759a1ea5e8a051dd4fb6e5a"}
Sep 15 12:41:08.392183 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.392147    2586 scope.go:117] "RemoveContainer" containerID="f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40"
Sep 15 12:41:08.393641 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.393589    2586 generic.go:356] "Generic (PLEG): container finished" podID="1a198470-3d87-4562-8839-71a990a5031e" containerID="7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55" exitCode=0
Sep 15 12:41:08.393641 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.393637    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"
Sep 15 12:41:08.393798 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.393683    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h" event={"ID":"1a198470-3d87-4562-8839-71a990a5031e","Type":"ContainerDied","Data":"7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55"}
Sep 15 12:41:08.393798 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.393753    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h" event={"ID":"1a198470-3d87-4562-8839-71a990a5031e","Type":"ContainerDied","Data":"e59169b945a2769e89dfba4026464f5ea01a9bce6809da631ffbc65533215a3d"}
Sep 15 12:41:08.416875 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.416084    2586 scope.go:117] "RemoveContainer" containerID="f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40"
Sep 15 12:41:08.418844 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:41:08.417153    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40\": container with ID starting with f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40 not found: ID does not exist" containerID="f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40"
Sep 15 12:41:08.418844 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.417188    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40"} err="failed to get container status \"f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40\": rpc error: code = NotFound desc = could not find container \"f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40\": container with ID starting with f8123e2604f3c5a99337228c644d0b2147e5dad434a09795d73dfcf877e7eb40 not found: ID does not exist"
Sep 15 12:41:08.418844 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.417211    2586 scope.go:117] "RemoveContainer" containerID="7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55"
Sep 15 12:41:08.433010 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.432933    2586 scope.go:117] "RemoveContainer" containerID="7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55"
Sep 15 12:41:08.434146 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.434114    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"]
Sep 15 12:41:08.434328 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:41:08.434303    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55\": container with ID starting with 7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55 not found: ID does not exist" containerID="7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55"
Sep 15 12:41:08.434407 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.434339    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55"} err="failed to get container status \"7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55\": rpc error: code = NotFound desc = could not find container \"7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55\": container with ID starting with 7d824c4e7544aebbe5c63fdc5313b1f18dd351aff4854e083ef256e5d9e19a55 not found: ID does not exist"
Sep 15 12:41:08.436144 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.436123    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1459b7a2-547bc9dd4b-s6v7h"]
Sep 15 12:41:08.445083 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.445063    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"]
Sep 15 12:41:08.448333 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.448313    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1459b7a2-fb489b5b5-mzgb8"]
Sep 15 12:41:08.652979 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.652895    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a198470-3d87-4562-8839-71a990a5031e" path="/var/lib/kubelet/pods/1a198470-3d87-4562-8839-71a990a5031e/volumes"
Sep 15 12:41:08.653299 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:08.653285    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bedfbdac-c04a-4a41-a1cc-619a5475b094" path="/var/lib/kubelet/pods/bedfbdac-c04a-4a41-a1cc-619a5475b094/volumes"
Sep 15 12:41:15.416971 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.416941    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:41:15.437901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.437578    2586 generic.go:356] "Generic (PLEG): container finished" podID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerID="80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f" exitCode=0
Sep 15 12:41:15.437901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.437630    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" event={"ID":"4ecc6aac-33e1-41ad-9768-a65e489f9418","Type":"ContainerDied","Data":"80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f"}
Sep 15 12:41:15.437901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.437683    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"
Sep 15 12:41:15.437901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.437730    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj" event={"ID":"4ecc6aac-33e1-41ad-9768-a65e489f9418","Type":"ContainerDied","Data":"29dddbe9f9bf10210d8384add2d086531ce62ded4116ad4cd859ca98ae39cde8"}
Sep 15 12:41:15.437901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.437762    2586 scope.go:117] "RemoveContainer" containerID="80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f"
Sep 15 12:41:15.463634 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.463606    2586 scope.go:117] "RemoveContainer" containerID="8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89"
Sep 15 12:41:15.473220 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.473197    2586 scope.go:117] "RemoveContainer" containerID="80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f"
Sep 15 12:41:15.473530 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:41:15.473510    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f\": container with ID starting with 80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f not found: ID does not exist" containerID="80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f"
Sep 15 12:41:15.473614 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.473545    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f"} err="failed to get container status \"80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f\": rpc error: code = NotFound desc = could not find container \"80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f\": container with ID starting with 80cd7464bcc40512bdfa6023ea887a3c43628aa78a2b48bef40b97e9c921a21f not found: ID does not exist"
Sep 15 12:41:15.473614 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.473571    2586 scope.go:117] "RemoveContainer" containerID="8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89"
Sep 15 12:41:15.473923 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:41:15.473895    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89\": container with ID starting with 8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89 not found: ID does not exist" containerID="8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89"
Sep 15 12:41:15.473989 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.473931    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89"} err="failed to get container status \"8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89\": rpc error: code = NotFound desc = could not find container \"8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89\": container with ID starting with 8eca12630f1743ec1256865cb688fe4399b536c1244045823bad702ae302dd89 not found: ID does not exist"
Sep 15 12:41:15.508283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508257    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-socket\") pod \"4ecc6aac-33e1-41ad-9768-a65e489f9418\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") "
Sep 15 12:41:15.508419 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508291    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ecc6aac-33e1-41ad-9768-a65e489f9418-istiod-ca-cert\") pod \"4ecc6aac-33e1-41ad-9768-a65e489f9418\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") "
Sep 15 12:41:15.508419 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508314    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-data\") pod \"4ecc6aac-33e1-41ad-9768-a65e489f9418\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") "
Sep 15 12:41:15.508419 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508368    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-token\") pod \"4ecc6aac-33e1-41ad-9768-a65e489f9418\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") "
Sep 15 12:41:15.508419 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508387    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9q2v\" (UniqueName: \"kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-kube-api-access-m9q2v\") pod \"4ecc6aac-33e1-41ad-9768-a65e489f9418\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") "
Sep 15 12:41:15.508644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508444    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-certs\") pod \"4ecc6aac-33e1-41ad-9768-a65e489f9418\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") "
Sep 15 12:41:15.508644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508486    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-envoy\") pod \"4ecc6aac-33e1-41ad-9768-a65e489f9418\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") "
Sep 15 12:41:15.508644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508545    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ecc6aac-33e1-41ad-9768-a65e489f9418" (UID: "4ecc6aac-33e1-41ad-9768-a65e489f9418"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:41:15.508644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508558    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-podinfo\") pod \"4ecc6aac-33e1-41ad-9768-a65e489f9418\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") "
Sep 15 12:41:15.508644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508616    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-credential-socket\") pod \"4ecc6aac-33e1-41ad-9768-a65e489f9418\" (UID: \"4ecc6aac-33e1-41ad-9768-a65e489f9418\") "
Sep 15 12:41:15.508921 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508660    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ecc6aac-33e1-41ad-9768-a65e489f9418" (UID: "4ecc6aac-33e1-41ad-9768-a65e489f9418"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:41:15.508921 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.508766    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ecc6aac-33e1-41ad-9768-a65e489f9418-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ecc6aac-33e1-41ad-9768-a65e489f9418" (UID: "4ecc6aac-33e1-41ad-9768-a65e489f9418"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:41:15.509086 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.509060    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ecc6aac-33e1-41ad-9768-a65e489f9418" (UID: "4ecc6aac-33e1-41ad-9768-a65e489f9418"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:41:15.509159 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.509090    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:15.509159 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.509111    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ecc6aac-33e1-41ad-9768-a65e489f9418-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:15.509159 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.509129    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:15.509159 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.509107    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ecc6aac-33e1-41ad-9768-a65e489f9418" (UID: "4ecc6aac-33e1-41ad-9768-a65e489f9418"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:41:15.510641 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.510611    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ecc6aac-33e1-41ad-9768-a65e489f9418" (UID: "4ecc6aac-33e1-41ad-9768-a65e489f9418"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:41:15.510818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.510794    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ecc6aac-33e1-41ad-9768-a65e489f9418" (UID: "4ecc6aac-33e1-41ad-9768-a65e489f9418"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:41:15.511043 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.511019    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-kube-api-access-m9q2v" (OuterVolumeSpecName: "kube-api-access-m9q2v") pod "4ecc6aac-33e1-41ad-9768-a65e489f9418" (UID: "4ecc6aac-33e1-41ad-9768-a65e489f9418"). InnerVolumeSpecName "kube-api-access-m9q2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:41:15.511043 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.511033    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ecc6aac-33e1-41ad-9768-a65e489f9418" (UID: "4ecc6aac-33e1-41ad-9768-a65e489f9418"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:41:15.609962 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.609928    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:15.609962 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.609956    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:15.609962 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.609968    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:15.610173 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.609976    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9q2v\" (UniqueName: \"kubernetes.io/projected/4ecc6aac-33e1-41ad-9768-a65e489f9418-kube-api-access-m9q2v\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:15.610173 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.609986    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:15.610173 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.609994    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ecc6aac-33e1-41ad-9768-a65e489f9418-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:15.762991 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.762965    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"]
Sep 15 12:41:15.766801 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:15.766763    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-1459b7a2-openshift-default-5d57bbccbd-6vhwj"]
Sep 15 12:41:16.654071 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:16.654043    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" path="/var/lib/kubelet/pods/4ecc6aac-33e1-41ad-9768-a65e489f9418/volumes"
Sep 15 12:41:33.927691 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:33.924448    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"]
Sep 15 12:41:33.927691 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:33.924829    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy" containerID="cri-o://5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7" gracePeriod=30
Sep 15 12:41:34.667982 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.667947    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"]
Sep 15 12:41:34.668480 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.668461    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bedfbdac-c04a-4a41-a1cc-619a5475b094" containerName="payload-processing"
Sep 15 12:41:34.668537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.668482    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="bedfbdac-c04a-4a41-a1cc-619a5475b094" containerName="payload-processing"
Sep 15 12:41:34.668537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.668491    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy"
Sep 15 12:41:34.668537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.668496    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy"
Sep 15 12:41:34.668537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.668525    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a198470-3d87-4562-8839-71a990a5031e" containerName="payload-pre-processing"
Sep 15 12:41:34.668537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.668530    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a198470-3d87-4562-8839-71a990a5031e" containerName="payload-pre-processing"
Sep 15 12:41:34.668711 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.668587    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy"
Sep 15 12:41:34.668711 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.668596    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="bedfbdac-c04a-4a41-a1cc-619a5475b094" containerName="payload-processing"
Sep 15 12:41:34.668711 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.668605    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a198470-3d87-4562-8839-71a990a5031e" containerName="payload-pre-processing"
Sep 15 12:41:34.671572 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.671547    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.673872 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.673844    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-0f955e-openshift-default-dockercfg-6fptf\""
Sep 15 12:41:34.682727 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.682707    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"]
Sep 15 12:41:34.801445 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.801402    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-certs\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.801612 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.801474    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-socket\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.801612 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.801512    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-podinfo\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.801612 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.801593    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-token\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.801752 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.801622    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-envoy\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.801752 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.801648    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsgjm\" (UniqueName: \"kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-kube-api-access-bsgjm\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.801904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.801813    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-credential-socket\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.801904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.801881    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-data\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.802001 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.801921    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/264ed769-9373-42c3-a2b4-01b26ebb2a82-istiod-ca-cert\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.903284 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.903253    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-podinfo\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.903496 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.903414    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-token\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.903496 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.903443    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-envoy\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.903496 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.903467    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsgjm\" (UniqueName: \"kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-kube-api-access-bsgjm\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.903982 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.903541    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-credential-socket\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.903982 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.903590    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-data\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.903982 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.903622    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/264ed769-9373-42c3-a2b4-01b26ebb2a82-istiod-ca-cert\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.903982 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.903660    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-certs\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.903982 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.903711    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-socket\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.904242 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.904097    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-data\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.904484 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.904442    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-credential-socket\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.904635 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.904608    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/264ed769-9373-42c3-a2b4-01b26ebb2a82-istiod-ca-cert\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.904896 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.904850    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-certs\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.905467 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.905448    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-socket\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.906872 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.906848    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-podinfo\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.907993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.907951    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-envoy\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.912017 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.911968    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-token\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.912116 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.912090    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsgjm\" (UniqueName: \"kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-kube-api-access-bsgjm\") pod \"e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") " pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:34.982694 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:34.982664    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:35.260763 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:35.260686    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"]
Sep 15 12:41:35.263586 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:41:35.263557    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264ed769_9373_42c3_a2b4_01b26ebb2a82.slice/crio-8b060d37cb35bb31de7c118cdca3d1aa49f8e5b6393d63170d0c8e1be53ed2ad WatchSource:0}: Error finding container 8b060d37cb35bb31de7c118cdca3d1aa49f8e5b6393d63170d0c8e1be53ed2ad: Status 404 returned error can't find the container with id 8b060d37cb35bb31de7c118cdca3d1aa49f8e5b6393d63170d0c8e1be53ed2ad
Sep 15 12:41:35.266099 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:35.266058    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:41:35.266173 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:35.266153    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:41:35.266224 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:35.266203    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:41:35.557390 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:35.557290    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" event={"ID":"264ed769-9373-42c3-a2b4-01b26ebb2a82","Type":"ContainerStarted","Data":"10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937"}
Sep 15 12:41:35.557390 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:35.557364    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" event={"ID":"264ed769-9373-42c3-a2b4-01b26ebb2a82","Type":"ContainerStarted","Data":"8b060d37cb35bb31de7c118cdca3d1aa49f8e5b6393d63170d0c8e1be53ed2ad"}
Sep 15 12:41:35.575689 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:35.575638    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" podStartSLOduration=1.575617856 podStartE2EDuration="1.575617856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:41:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:41:35.575222889 +0000 UTC m=+963.387134332" watchObservedRunningTime="2026-09-15 12:41:35.575617856 +0000 UTC m=+963.387529299"
Sep 15 12:41:35.983686 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:35.983657    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:36.515615 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:36.515573    2586 patch_prober.go:28] interesting pod/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 12:41:36.515828 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:36.515635    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 12:41:36.988316 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:36.988284    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:37.569425 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:37.569385    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:37.570241 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:37.570221    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:41:39.096915 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.096887    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:41:39.252218 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252182    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5d76\" (UniqueName: \"kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-kube-api-access-g5d76\") pod \"480d35db-c411-41d1-815c-5dbb1e69a65b\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") "
Sep 15 12:41:39.252417 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252237    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-data\") pod \"480d35db-c411-41d1-815c-5dbb1e69a65b\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") "
Sep 15 12:41:39.252417 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252299    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-certs\") pod \"480d35db-c411-41d1-815c-5dbb1e69a65b\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") "
Sep 15 12:41:39.252417 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252336    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/480d35db-c411-41d1-815c-5dbb1e69a65b-istiod-ca-cert\") pod \"480d35db-c411-41d1-815c-5dbb1e69a65b\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") "
Sep 15 12:41:39.252417 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252393    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-podinfo\") pod \"480d35db-c411-41d1-815c-5dbb1e69a65b\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") "
Sep 15 12:41:39.252638 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252431    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-socket\") pod \"480d35db-c411-41d1-815c-5dbb1e69a65b\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") "
Sep 15 12:41:39.252638 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252474    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-envoy\") pod \"480d35db-c411-41d1-815c-5dbb1e69a65b\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") "
Sep 15 12:41:39.252638 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252501    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-credential-socket\") pod \"480d35db-c411-41d1-815c-5dbb1e69a65b\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") "
Sep 15 12:41:39.252638 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252557    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-token\") pod \"480d35db-c411-41d1-815c-5dbb1e69a65b\" (UID: \"480d35db-c411-41d1-815c-5dbb1e69a65b\") "
Sep 15 12:41:39.252638 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252610    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "480d35db-c411-41d1-815c-5dbb1e69a65b" (UID: "480d35db-c411-41d1-815c-5dbb1e69a65b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:41:39.252638 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252623    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-data" (OuterVolumeSpecName: "istio-data") pod "480d35db-c411-41d1-815c-5dbb1e69a65b" (UID: "480d35db-c411-41d1-815c-5dbb1e69a65b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:41:39.252943 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252719    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/480d35db-c411-41d1-815c-5dbb1e69a65b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "480d35db-c411-41d1-815c-5dbb1e69a65b" (UID: "480d35db-c411-41d1-815c-5dbb1e69a65b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:41:39.252943 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252822    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "480d35db-c411-41d1-815c-5dbb1e69a65b" (UID: "480d35db-c411-41d1-815c-5dbb1e69a65b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:41:39.253062 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252930    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "480d35db-c411-41d1-815c-5dbb1e69a65b" (UID: "480d35db-c411-41d1-815c-5dbb1e69a65b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:41:39.253062 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.252986    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:39.253062 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.253006    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/480d35db-c411-41d1-815c-5dbb1e69a65b-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:39.253062 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.253020    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:39.253062 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.253031    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:39.254535 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.254490    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-kube-api-access-g5d76" (OuterVolumeSpecName: "kube-api-access-g5d76") pod "480d35db-c411-41d1-815c-5dbb1e69a65b" (UID: "480d35db-c411-41d1-815c-5dbb1e69a65b"). InnerVolumeSpecName "kube-api-access-g5d76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:41:39.254666 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.254586    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "480d35db-c411-41d1-815c-5dbb1e69a65b" (UID: "480d35db-c411-41d1-815c-5dbb1e69a65b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:41:39.255061 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.255038    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-token" (OuterVolumeSpecName: "istio-token") pod "480d35db-c411-41d1-815c-5dbb1e69a65b" (UID: "480d35db-c411-41d1-815c-5dbb1e69a65b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:41:39.255134 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.255055    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "480d35db-c411-41d1-815c-5dbb1e69a65b" (UID: "480d35db-c411-41d1-815c-5dbb1e69a65b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:41:39.353868 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.353836    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:39.353868 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.353863    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5d76\" (UniqueName: \"kubernetes.io/projected/480d35db-c411-41d1-815c-5dbb1e69a65b-kube-api-access-g5d76\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:39.353868 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.353873    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:39.354078 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.353881    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:39.354078 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.353889    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d35db-c411-41d1-815c-5dbb1e69a65b-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:41:39.588011 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.587927    2586 generic.go:356] "Generic (PLEG): container finished" podID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerID="5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7" exitCode=0
Sep 15 12:41:39.588011 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.587990    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"
Sep 15 12:41:39.588196 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.588013    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" event={"ID":"480d35db-c411-41d1-815c-5dbb1e69a65b","Type":"ContainerDied","Data":"5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7"}
Sep 15 12:41:39.588196 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.588104    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln" event={"ID":"480d35db-c411-41d1-815c-5dbb1e69a65b","Type":"ContainerDied","Data":"2056fad6948d25f43579ce3f7f57e8857dc1afb96bbe38143ea8ff4f631cd3ff"}
Sep 15 12:41:39.588196 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.588133    2586 scope.go:117] "RemoveContainer" containerID="5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7"
Sep 15 12:41:39.598980 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.598957    2586 scope.go:117] "RemoveContainer" containerID="d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317"
Sep 15 12:41:39.608592 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.608562    2586 scope.go:117] "RemoveContainer" containerID="5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7"
Sep 15 12:41:39.609153 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:41:39.609130    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7\": container with ID starting with 5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7 not found: ID does not exist" containerID="5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7"
Sep 15 12:41:39.609237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.609158    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7"} err="failed to get container status \"5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7\": rpc error: code = NotFound desc = could not find container \"5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7\": container with ID starting with 5706e2440eabd5da9658d02c7384b2a580b4c0ccf6cd7b9c13c333fafb1d62b7 not found: ID does not exist"
Sep 15 12:41:39.609237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.609175    2586 scope.go:117] "RemoveContainer" containerID="d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317"
Sep 15 12:41:39.609478 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:41:39.609456    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317\": container with ID starting with d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317 not found: ID does not exist" containerID="d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317"
Sep 15 12:41:39.609516 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.609488    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317"} err="failed to get container status \"d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317\": rpc error: code = NotFound desc = could not find container \"d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317\": container with ID starting with d69fb9599ce109111a9e15dee46c64982daa6ff35e23e07ab8cd7e774d2f4317 not found: ID does not exist"
Sep 15 12:41:39.614463 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.614439    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"]
Sep 15 12:41:39.616422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:39.616402    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-bf37f4-openshift-default-8664fd7466-69gln"]
Sep 15 12:41:40.653365 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:40.653332    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" path="/var/lib/kubelet/pods/480d35db-c411-41d1-815c-5dbb1e69a65b/volumes"
Sep 15 12:41:41.266219 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.266183    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"]
Sep 15 12:41:41.266797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.266756    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy"
Sep 15 12:41:41.266797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.266789    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy"
Sep 15 12:41:41.266954 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.266816    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy"
Sep 15 12:41:41.266954 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.266822    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy"
Sep 15 12:41:41.266954 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.266829    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy"
Sep 15 12:41:41.266954 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.266836    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy"
Sep 15 12:41:41.266954 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.266903    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy"
Sep 15 12:41:41.266954 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.266913    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ecc6aac-33e1-41ad-9768-a65e489f9418" containerName="istio-proxy"
Sep 15 12:41:41.269667 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.269650    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:41:41.271993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.271970    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-0f955e\""
Sep 15 12:41:41.272110 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.272089    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-0f955e-dockercfg-5s5hz\""
Sep 15 12:41:41.282396 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.282373    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"]
Sep 15 12:41:41.307268 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.307238    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"]
Sep 15 12:41:41.307716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.307703    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="480d35db-c411-41d1-815c-5dbb1e69a65b" containerName="istio-proxy"
Sep 15 12:41:41.309592 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.309576    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:41:41.320461 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.320439    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"]
Sep 15 12:41:41.374506 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.374469    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmvkx\" (UniqueName: \"kubernetes.io/projected/0a6306f5-010e-4602-b70c-02b61c715ec8-kube-api-access-xmvkx\") pod \"payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q\" (UID: \"0a6306f5-010e-4602-b70c-02b61c715ec8\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:41:41.374813 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.374756    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6306f5-010e-4602-b70c-02b61c715ec8-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q\" (UID: \"0a6306f5-010e-4602-b70c-02b61c715ec8\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:41:41.475927 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.475894    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmvkx\" (UniqueName: \"kubernetes.io/projected/0a6306f5-010e-4602-b70c-02b61c715ec8-kube-api-access-xmvkx\") pod \"payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q\" (UID: \"0a6306f5-010e-4602-b70c-02b61c715ec8\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:41:41.476109 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.475941    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj9nh\" (UniqueName: \"kubernetes.io/projected/81a2390c-250f-4448-b166-48fa5e439d84-kube-api-access-zj9nh\") pod \"payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4\" (UID: \"81a2390c-250f-4448-b166-48fa5e439d84\") " pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:41:41.476109 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.475963    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6306f5-010e-4602-b70c-02b61c715ec8-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q\" (UID: \"0a6306f5-010e-4602-b70c-02b61c715ec8\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:41:41.476109 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.475990    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81a2390c-250f-4448-b166-48fa5e439d84-plugins-config-volume\") pod \"payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4\" (UID: \"81a2390c-250f-4448-b166-48fa5e439d84\") " pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:41:41.476701 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.476682    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6306f5-010e-4602-b70c-02b61c715ec8-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q\" (UID: \"0a6306f5-010e-4602-b70c-02b61c715ec8\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:41:41.483690 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.483668    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmvkx\" (UniqueName: \"kubernetes.io/projected/0a6306f5-010e-4602-b70c-02b61c715ec8-kube-api-access-xmvkx\") pod \"payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q\" (UID: \"0a6306f5-010e-4602-b70c-02b61c715ec8\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:41:41.576751 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.576674    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81a2390c-250f-4448-b166-48fa5e439d84-plugins-config-volume\") pod \"payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4\" (UID: \"81a2390c-250f-4448-b166-48fa5e439d84\") " pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:41:41.576904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.576833    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj9nh\" (UniqueName: \"kubernetes.io/projected/81a2390c-250f-4448-b166-48fa5e439d84-kube-api-access-zj9nh\") pod \"payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4\" (UID: \"81a2390c-250f-4448-b166-48fa5e439d84\") " pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:41:41.577377 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.577359    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81a2390c-250f-4448-b166-48fa5e439d84-plugins-config-volume\") pod \"payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4\" (UID: \"81a2390c-250f-4448-b166-48fa5e439d84\") " pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:41:41.581536 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.581519    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:41:41.587631 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.587612    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj9nh\" (UniqueName: \"kubernetes.io/projected/81a2390c-250f-4448-b166-48fa5e439d84-kube-api-access-zj9nh\") pod \"payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4\" (UID: \"81a2390c-250f-4448-b166-48fa5e439d84\") " pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:41:41.620307 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.620278    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:41:41.861361 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.861334    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"]
Sep 15 12:41:41.862997 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:41:41.862973    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6306f5_010e_4602_b70c_02b61c715ec8.slice/crio-f187c01268da4ba530458899caad38413c6b61afb39e33432fdb7484e2396ba5 WatchSource:0}: Error finding container f187c01268da4ba530458899caad38413c6b61afb39e33432fdb7484e2396ba5: Status 404 returned error can't find the container with id f187c01268da4ba530458899caad38413c6b61afb39e33432fdb7484e2396ba5
Sep 15 12:41:41.927170 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:41.927146    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"]
Sep 15 12:41:41.928360 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:41:41.928338    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a2390c_250f_4448_b166_48fa5e439d84.slice/crio-1ca9d2bf8d104cde8af2349d861373a431bac7356b42473336a98461ed2a6ddc WatchSource:0}: Error finding container 1ca9d2bf8d104cde8af2349d861373a431bac7356b42473336a98461ed2a6ddc: Status 404 returned error can't find the container with id 1ca9d2bf8d104cde8af2349d861373a431bac7356b42473336a98461ed2a6ddc
Sep 15 12:41:42.610171 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:42.610137    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q" event={"ID":"0a6306f5-010e-4602-b70c-02b61c715ec8","Type":"ContainerStarted","Data":"02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd"}
Sep 15 12:41:42.610370 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:42.610192    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q" event={"ID":"0a6306f5-010e-4602-b70c-02b61c715ec8","Type":"ContainerStarted","Data":"f187c01268da4ba530458899caad38413c6b61afb39e33432fdb7484e2396ba5"}
Sep 15 12:41:42.610370 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:42.610236    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:41:42.611876 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:42.611849    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4" event={"ID":"81a2390c-250f-4448-b166-48fa5e439d84","Type":"ContainerStarted","Data":"d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd"}
Sep 15 12:41:42.611990 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:42.611897    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4" event={"ID":"81a2390c-250f-4448-b166-48fa5e439d84","Type":"ContainerStarted","Data":"1ca9d2bf8d104cde8af2349d861373a431bac7356b42473336a98461ed2a6ddc"}
Sep 15 12:41:42.611990 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:42.611936    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:41:42.625730 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:42.625671    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q" podStartSLOduration=1.625651558 podStartE2EDuration="1.625651558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:41:42.625093736 +0000 UTC m=+970.437005180" watchObservedRunningTime="2026-09-15 12:41:42.625651558 +0000 UTC m=+970.437563001"
Sep 15 12:41:42.643162 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:42.643107    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4" podStartSLOduration=1.6430901759999998 podStartE2EDuration="1.643090176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:41:42.639529537 +0000 UTC m=+970.451440980" watchObservedRunningTime="2026-09-15 12:41:42.643090176 +0000 UTC m=+970.455001620"
Sep 15 12:41:53.622008 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:53.621960    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:41:53.622981 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:41:53.622957    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:42:02.735365 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:02.735331    2586 generic.go:356] "Generic (PLEG): container finished" podID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerID="10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937" exitCode=0
Sep 15 12:42:02.735910 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:02.735414    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" event={"ID":"264ed769-9373-42c3-a2b4-01b26ebb2a82","Type":"ContainerDied","Data":"10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937"}
Sep 15 12:42:02.736015 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:02.735995    2586 scope.go:117] "RemoveContainer" containerID="10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937"
Sep 15 12:42:02.737188 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:02.737160    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:42:02.737265 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:02.737232    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:42:02.737309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:02.737265    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:42:03.742822 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:03.742774    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" event={"ID":"264ed769-9373-42c3-a2b4-01b26ebb2a82","Type":"ContainerStarted","Data":"99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137"}
Sep 15 12:42:03.983836 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:03.983774    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:42:03.985212 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:03.985187    2586 patch_prober.go:28] interesting pod/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 15 12:42:03.985323 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:03.985236    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 15 12:42:04.984000 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:04.983966    2586 patch_prober.go:28] interesting pod/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 15 12:42:04.984385 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:04.984037    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 15 12:42:05.983690 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:05.983652    2586 patch_prober.go:28] interesting pod/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 15 12:42:05.983879 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:05.983706    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 15 12:42:06.983131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:06.983098    2586 patch_prober.go:28] interesting pod/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 15 12:42:06.983517 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:06.983151    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 15 12:42:07.569897 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:07.569862    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:42:07.983078 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:07.983046    2586 patch_prober.go:28] interesting pod/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 15 12:42:07.983259 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:07.983106    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 15 12:42:08.987339 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:08.987301    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:42:08.988502 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:08.988482    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:42:52.176427 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.176390    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"]
Sep 15 12:42:52.181264 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.181240    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.183433 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.183409    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b7e2d651-serving-cert\""
Sep 15 12:42:52.189159 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.189132    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"]
Sep 15 12:42:52.297971 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.297935    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b94f95-1f92-412f-b614-a525ff3fa9f4-openshift-service-ca\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.298183 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.298001    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-maas-api-tls\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.298183 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.298166    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9nbx\" (UniqueName: \"kubernetes.io/projected/a1b94f95-1f92-412f-b614-a525ff3fa9f4-kube-api-access-g9nbx\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.298318 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.298206    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-metrics-certs\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.399139 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.399111    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-metrics-certs\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.399386 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.399187    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b94f95-1f92-412f-b614-a525ff3fa9f4-openshift-service-ca\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.399386 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.399227    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-maas-api-tls\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.399386 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.399321    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9nbx\" (UniqueName: \"kubernetes.io/projected/a1b94f95-1f92-412f-b614-a525ff3fa9f4-kube-api-access-g9nbx\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.399559 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:42:52.399418    2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-serving-cert: secret "maas-api-e2e-mt-b7e2d651-serving-cert" not found
Sep 15 12:42:52.399559 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:42:52.399484    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-maas-api-tls podName:a1b94f95-1f92-412f-b614-a525ff3fa9f4 nodeName:}" failed. No retries permitted until 2026-09-15 12:42:52.89946398 +0000 UTC m=+1040.711375413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-maas-api-tls") pod "maas-api-e2e-mt-b7e2d651-96c464df8-cdggl" (UID: "a1b94f95-1f92-412f-b614-a525ff3fa9f4") : secret "maas-api-e2e-mt-b7e2d651-serving-cert" not found
Sep 15 12:42:52.400074 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.400044    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b94f95-1f92-412f-b614-a525ff3fa9f4-openshift-service-ca\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.402114 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.402090    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-metrics-certs\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.408011 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.407989    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9nbx\" (UniqueName: \"kubernetes.io/projected/a1b94f95-1f92-412f-b614-a525ff3fa9f4-kube-api-access-g9nbx\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.906305 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.906273    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-maas-api-tls\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:52.911048 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:52.910966    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-maas-api-tls\") pod \"maas-api-e2e-mt-b7e2d651-96c464df8-cdggl\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:53.094734 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:53.094704    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:53.366958 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:53.366415    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"]
Sep 15 12:42:53.369531 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:42:53.369497    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b94f95_1f92_412f_b614_a525ff3fa9f4.slice/crio-056d287b23f8dad6258a442416715e3f1ca2f61eb7db9346756d36bb971a4791 WatchSource:0}: Error finding container 056d287b23f8dad6258a442416715e3f1ca2f61eb7db9346756d36bb971a4791: Status 404 returned error can't find the container with id 056d287b23f8dad6258a442416715e3f1ca2f61eb7db9346756d36bb971a4791
Sep 15 12:42:53.371609 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:53.371586    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 12:42:54.023854 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:54.023823    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl" event={"ID":"a1b94f95-1f92-412f-b614-a525ff3fa9f4","Type":"ContainerStarted","Data":"fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465"}
Sep 15 12:42:54.024027 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:54.023873    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl" event={"ID":"a1b94f95-1f92-412f-b614-a525ff3fa9f4","Type":"ContainerStarted","Data":"056d287b23f8dad6258a442416715e3f1ca2f61eb7db9346756d36bb971a4791"}
Sep 15 12:42:54.024027 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:54.023963    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:42:54.041720 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:42:54.041630    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl" podStartSLOduration=1.631061883 podStartE2EDuration="2.0416159s" totalImagesPullingTime="410.554017ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:42:54.04018724 +0000 UTC m=+1041.852098683" watchObservedRunningTime="2026-09-15 12:42:54.0416159 +0000 UTC m=+1041.853527342"
Sep 15 12:43:00.035261 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:00.035224    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:43:21.038819 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.038640    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"]
Sep 15 12:43:21.041695 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.041674    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.044445 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.044424    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2e3fef\"/\"kube-root-ca.crt\""
Sep 15 12:43:21.045879 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.045854    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2e3fef\"/\"default-dockercfg-4jcrj\""
Sep 15 12:43:21.045985 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.045961    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2e3fef\"/\"test-model-2e3fef-kserve-self-signed-certs\""
Sep 15 12:43:21.045985 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.045972    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2e3fef\"/\"openshift-service-ca.crt\""
Sep 15 12:43:21.054496 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.054474    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"]
Sep 15 12:43:21.190994 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.190965    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tmp-dir\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.191138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.190998    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tls-certs\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.191138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.191076    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-model-cache\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.191138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.191127    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-dshm\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.191251 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.191154    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-home\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.191251 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.191175    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z5tl\" (UniqueName: \"kubernetes.io/projected/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-kube-api-access-4z5tl\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.291892 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.291759    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-dshm\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.291892 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.291876    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-home\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.292133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.291918    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z5tl\" (UniqueName: \"kubernetes.io/projected/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-kube-api-access-4z5tl\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.292133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.291987    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tmp-dir\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.292133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.292015    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tls-certs\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.292133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.292117    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-model-cache\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.292372 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.292292    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-home\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.292372 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.292342    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tmp-dir\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.292487 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.292463    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-model-cache\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.294009 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.293989    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-dshm\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.294478 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.294459    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tls-certs\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.300816 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.300766    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z5tl\" (UniqueName: \"kubernetes.io/projected/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-kube-api-access-4z5tl\") pod \"test-model-2e3fef-kserve-84b594497f-858lg\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") " pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.352033 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.352012    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:21.627150 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:21.627120    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"]
Sep 15 12:43:21.628527 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:43:21.628498    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06dee008_23b7_425b_bc8d_7f08ddc9c9ac.slice/crio-67cba2f358ea22f7fe0fcffa41a3972cc38a412eb10044023b0e3ea6274b6c3a WatchSource:0}: Error finding container 67cba2f358ea22f7fe0fcffa41a3972cc38a412eb10044023b0e3ea6274b6c3a: Status 404 returned error can't find the container with id 67cba2f358ea22f7fe0fcffa41a3972cc38a412eb10044023b0e3ea6274b6c3a
Sep 15 12:43:22.174008 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:22.173972    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg" event={"ID":"06dee008-23b7-425b-bc8d-7f08ddc9c9ac","Type":"ContainerStarted","Data":"45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb"}
Sep 15 12:43:22.174008 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:22.174011    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg" event={"ID":"06dee008-23b7-425b-bc8d-7f08ddc9c9ac","Type":"ContainerStarted","Data":"67cba2f358ea22f7fe0fcffa41a3972cc38a412eb10044023b0e3ea6274b6c3a"}
Sep 15 12:43:22.191085 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:22.191039    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg" podStartSLOduration=1.191024409 podStartE2EDuration="1.191024409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:43:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:43:22.190059987 +0000 UTC m=+1070.001971430" watchObservedRunningTime="2026-09-15 12:43:22.191024409 +0000 UTC m=+1070.002935850"
Sep 15 12:43:31.352895 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:31.352856    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:31.353357 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:31.352913    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:31.365473 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:31.365451    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:32.242617 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:32.242587    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:43:35.346295 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.346259    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"]
Sep 15 12:43:35.346711 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.346560    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4" podUID="81a2390c-250f-4448-b166-48fa5e439d84" containerName="payload-processing" containerID="cri-o://d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd" gracePeriod=30
Sep 15 12:43:35.356887 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.356859    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"]
Sep 15 12:43:35.357111 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.357089    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q" podUID="0a6306f5-010e-4602-b70c-02b61c715ec8" containerName="payload-pre-processing" containerID="cri-o://02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd" gracePeriod=30
Sep 15 12:43:35.624985 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.624960    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:43:35.655376 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.655340    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:43:35.731340 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.731309    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6306f5-010e-4602-b70c-02b61c715ec8-plugins-config-volume\") pod \"0a6306f5-010e-4602-b70c-02b61c715ec8\" (UID: \"0a6306f5-010e-4602-b70c-02b61c715ec8\") "
Sep 15 12:43:35.731515 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.731423    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmvkx\" (UniqueName: \"kubernetes.io/projected/0a6306f5-010e-4602-b70c-02b61c715ec8-kube-api-access-xmvkx\") pod \"0a6306f5-010e-4602-b70c-02b61c715ec8\" (UID: \"0a6306f5-010e-4602-b70c-02b61c715ec8\") "
Sep 15 12:43:35.731515 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.731481    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81a2390c-250f-4448-b166-48fa5e439d84-plugins-config-volume\") pod \"81a2390c-250f-4448-b166-48fa5e439d84\" (UID: \"81a2390c-250f-4448-b166-48fa5e439d84\") "
Sep 15 12:43:35.731634 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.731521    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zj9nh\" (UniqueName: \"kubernetes.io/projected/81a2390c-250f-4448-b166-48fa5e439d84-kube-api-access-zj9nh\") pod \"81a2390c-250f-4448-b166-48fa5e439d84\" (UID: \"81a2390c-250f-4448-b166-48fa5e439d84\") "
Sep 15 12:43:35.731690 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.731628    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a6306f5-010e-4602-b70c-02b61c715ec8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a6306f5-010e-4602-b70c-02b61c715ec8" (UID: "0a6306f5-010e-4602-b70c-02b61c715ec8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:43:35.731863 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.731835    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6306f5-010e-4602-b70c-02b61c715ec8-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:43:35.732023 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.731861    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81a2390c-250f-4448-b166-48fa5e439d84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "81a2390c-250f-4448-b166-48fa5e439d84" (UID: "81a2390c-250f-4448-b166-48fa5e439d84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:43:35.733609 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.733584    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81a2390c-250f-4448-b166-48fa5e439d84-kube-api-access-zj9nh" (OuterVolumeSpecName: "kube-api-access-zj9nh") pod "81a2390c-250f-4448-b166-48fa5e439d84" (UID: "81a2390c-250f-4448-b166-48fa5e439d84"). InnerVolumeSpecName "kube-api-access-zj9nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:43:35.733609 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.733598    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a6306f5-010e-4602-b70c-02b61c715ec8-kube-api-access-xmvkx" (OuterVolumeSpecName: "kube-api-access-xmvkx") pod "0a6306f5-010e-4602-b70c-02b61c715ec8" (UID: "0a6306f5-010e-4602-b70c-02b61c715ec8"). InnerVolumeSpecName "kube-api-access-xmvkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:43:35.832542 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.832511    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmvkx\" (UniqueName: \"kubernetes.io/projected/0a6306f5-010e-4602-b70c-02b61c715ec8-kube-api-access-xmvkx\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:43:35.832542 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.832540    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81a2390c-250f-4448-b166-48fa5e439d84-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:43:35.832698 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:35.832553    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zj9nh\" (UniqueName: \"kubernetes.io/projected/81a2390c-250f-4448-b166-48fa5e439d84-kube-api-access-zj9nh\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:43:36.260678 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.260645    2586 generic.go:356] "Generic (PLEG): container finished" podID="81a2390c-250f-4448-b166-48fa5e439d84" containerID="d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd" exitCode=0
Sep 15 12:43:36.260678 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.260678    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4" event={"ID":"81a2390c-250f-4448-b166-48fa5e439d84","Type":"ContainerDied","Data":"d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd"}
Sep 15 12:43:36.260678 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.260702    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"
Sep 15 12:43:36.261063 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.260748    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4" event={"ID":"81a2390c-250f-4448-b166-48fa5e439d84","Type":"ContainerDied","Data":"1ca9d2bf8d104cde8af2349d861373a431bac7356b42473336a98461ed2a6ddc"}
Sep 15 12:43:36.261063 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.260803    2586 scope.go:117] "RemoveContainer" containerID="d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd"
Sep 15 12:43:36.262325 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.262303    2586 generic.go:356] "Generic (PLEG): container finished" podID="0a6306f5-010e-4602-b70c-02b61c715ec8" containerID="02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd" exitCode=0
Sep 15 12:43:36.262422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.262333    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"
Sep 15 12:43:36.262422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.262367    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q" event={"ID":"0a6306f5-010e-4602-b70c-02b61c715ec8","Type":"ContainerDied","Data":"02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd"}
Sep 15 12:43:36.262422 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.262400    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q" event={"ID":"0a6306f5-010e-4602-b70c-02b61c715ec8","Type":"ContainerDied","Data":"f187c01268da4ba530458899caad38413c6b61afb39e33432fdb7484e2396ba5"}
Sep 15 12:43:36.272524 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.272500    2586 scope.go:117] "RemoveContainer" containerID="d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd"
Sep 15 12:43:36.272829 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:43:36.272806    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd\": container with ID starting with d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd not found: ID does not exist" containerID="d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd"
Sep 15 12:43:36.272901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.272838    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd"} err="failed to get container status \"d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd\": rpc error: code = NotFound desc = could not find container \"d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd\": container with ID starting with d5216709db4a40cc7159925b3b4c1db08ced26e96ecb00b2814b281ea10a14dd not found: ID does not exist"
Sep 15 12:43:36.272901 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.272855    2586 scope.go:117] "RemoveContainer" containerID="02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd"
Sep 15 12:43:36.281138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.281117    2586 scope.go:117] "RemoveContainer" containerID="02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd"
Sep 15 12:43:36.281996 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:43:36.281592    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd\": container with ID starting with 02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd not found: ID does not exist" containerID="02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd"
Sep 15 12:43:36.281996 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.281624    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd"} err="failed to get container status \"02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd\": rpc error: code = NotFound desc = could not find container \"02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd\": container with ID starting with 02de683caddd91cee46d593b3adea25e72b167e4117c4b50d4e3a252f27c4dbd not found: ID does not exist"
Sep 15 12:43:36.283920 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.283898    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"]
Sep 15 12:43:36.287468 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.287447    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0f955e-79f5c9f465-x5wd4"]
Sep 15 12:43:36.296631 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.296599    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"]
Sep 15 12:43:36.300976 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.300953    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0f955e-79d49c496b-r8z8q"]
Sep 15 12:43:36.652823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.652733    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a6306f5-010e-4602-b70c-02b61c715ec8" path="/var/lib/kubelet/pods/0a6306f5-010e-4602-b70c-02b61c715ec8/volumes"
Sep 15 12:43:36.653177 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:36.653120    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81a2390c-250f-4448-b166-48fa5e439d84" path="/var/lib/kubelet/pods/81a2390c-250f-4448-b166-48fa5e439d84/volumes"
Sep 15 12:43:39.445928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.445897    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"]
Sep 15 12:43:39.446374 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.446143    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl" podUID="a1b94f95-1f92-412f-b614-a525ff3fa9f4" containerName="maas-api" containerID="cri-o://fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465" gracePeriod=30
Sep 15 12:43:39.706228 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.706170    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:43:39.773324 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.773300    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b94f95-1f92-412f-b614-a525ff3fa9f4-openshift-service-ca\") pod \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") "
Sep 15 12:43:39.773455 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.773403    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9nbx\" (UniqueName: \"kubernetes.io/projected/a1b94f95-1f92-412f-b614-a525ff3fa9f4-kube-api-access-g9nbx\") pod \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") "
Sep 15 12:43:39.773455 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.773429    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-metrics-certs\") pod \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") "
Sep 15 12:43:39.773455 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.773452    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-maas-api-tls\") pod \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\" (UID: \"a1b94f95-1f92-412f-b614-a525ff3fa9f4\") "
Sep 15 12:43:39.773686 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.773658    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b94f95-1f92-412f-b614-a525ff3fa9f4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1b94f95-1f92-412f-b614-a525ff3fa9f4" (UID: "a1b94f95-1f92-412f-b614-a525ff3fa9f4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:43:39.773836 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.773820    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b94f95-1f92-412f-b614-a525ff3fa9f4-openshift-service-ca\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:43:39.775672 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.775645    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a1b94f95-1f92-412f-b614-a525ff3fa9f4" (UID: "a1b94f95-1f92-412f-b614-a525ff3fa9f4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:43:39.775796 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.775713    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1b94f95-1f92-412f-b614-a525ff3fa9f4" (UID: "a1b94f95-1f92-412f-b614-a525ff3fa9f4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:43:39.775796 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.775752    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b94f95-1f92-412f-b614-a525ff3fa9f4-kube-api-access-g9nbx" (OuterVolumeSpecName: "kube-api-access-g9nbx") pod "a1b94f95-1f92-412f-b614-a525ff3fa9f4" (UID: "a1b94f95-1f92-412f-b614-a525ff3fa9f4"). InnerVolumeSpecName "kube-api-access-g9nbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:43:39.875311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.875276    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9nbx\" (UniqueName: \"kubernetes.io/projected/a1b94f95-1f92-412f-b614-a525ff3fa9f4-kube-api-access-g9nbx\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:43:39.875311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.875306    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:43:39.875311 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:39.875316    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1b94f95-1f92-412f-b614-a525ff3fa9f4-maas-api-tls\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:43:40.287543 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:40.287515    2586 generic.go:356] "Generic (PLEG): container finished" podID="a1b94f95-1f92-412f-b614-a525ff3fa9f4" containerID="fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465" exitCode=0
Sep 15 12:43:40.287709 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:40.287560    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"
Sep 15 12:43:40.287709 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:40.287573    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl" event={"ID":"a1b94f95-1f92-412f-b614-a525ff3fa9f4","Type":"ContainerDied","Data":"fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465"}
Sep 15 12:43:40.287709 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:40.287639    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl" event={"ID":"a1b94f95-1f92-412f-b614-a525ff3fa9f4","Type":"ContainerDied","Data":"056d287b23f8dad6258a442416715e3f1ca2f61eb7db9346756d36bb971a4791"}
Sep 15 12:43:40.287709 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:40.287661    2586 scope.go:117] "RemoveContainer" containerID="fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465"
Sep 15 12:43:40.298902 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:40.298820    2586 scope.go:117] "RemoveContainer" containerID="fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465"
Sep 15 12:43:40.300365 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:43:40.300339    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465\": container with ID starting with fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465 not found: ID does not exist" containerID="fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465"
Sep 15 12:43:40.300542 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:40.300512    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465"} err="failed to get container status \"fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465\": rpc error: code = NotFound desc = could not find container \"fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465\": container with ID starting with fb3397be9ba61557b3ecf33aeb03082a0954f0cf970e5a878bf43362862e8465 not found: ID does not exist"
Sep 15 12:43:40.309569 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:40.309548    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"]
Sep 15 12:43:40.313447 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:40.313423    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e2d651-96c464df8-cdggl"]
Sep 15 12:43:40.655838 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:43:40.655752    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1b94f95-1f92-412f-b614-a525ff3fa9f4" path="/var/lib/kubelet/pods/a1b94f95-1f92-412f-b614-a525ff3fa9f4/volumes"
Sep 15 12:44:07.875181 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.875151    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"]
Sep 15 12:44:07.875690 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.875653    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1b94f95-1f92-412f-b614-a525ff3fa9f4" containerName="maas-api"
Sep 15 12:44:07.875690 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.875670    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1b94f95-1f92-412f-b614-a525ff3fa9f4" containerName="maas-api"
Sep 15 12:44:07.875769 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.875697    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6306f5-010e-4602-b70c-02b61c715ec8" containerName="payload-pre-processing"
Sep 15 12:44:07.875769 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.875703    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6306f5-010e-4602-b70c-02b61c715ec8" containerName="payload-pre-processing"
Sep 15 12:44:07.875769 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.875710    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a2390c-250f-4448-b166-48fa5e439d84" containerName="payload-processing"
Sep 15 12:44:07.875769 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.875715    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a2390c-250f-4448-b166-48fa5e439d84" containerName="payload-processing"
Sep 15 12:44:07.875925 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.875791    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a2390c-250f-4448-b166-48fa5e439d84" containerName="payload-processing"
Sep 15 12:44:07.875925 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.875802    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6306f5-010e-4602-b70c-02b61c715ec8" containerName="payload-pre-processing"
Sep 15 12:44:07.875925 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.875807    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1b94f95-1f92-412f-b614-a525ff3fa9f4" containerName="maas-api"
Sep 15 12:44:07.877970 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.877954    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:44:07.880381 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.880363    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-a1ba7b-dockercfg-2twvj\""
Sep 15 12:44:07.880525 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.880413    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-a1ba7b\""
Sep 15 12:44:07.889342 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.889318    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"]
Sep 15 12:44:07.914515 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.914488    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"]
Sep 15 12:44:07.917077 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.917058    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:44:07.930786 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.930752    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"]
Sep 15 12:44:07.936562 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.936543    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fafcfc94-94c1-45ee-a507-c26613679560-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf\" (UID: \"fafcfc94-94c1-45ee-a507-c26613679560\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:44:07.936662 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.936582    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1f2c109-17c9-4045-8e75-aaf57109caf1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22\" (UID: \"a1f2c109-17c9-4045-8e75-aaf57109caf1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:44:07.936662 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.936651    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p84b7\" (UniqueName: \"kubernetes.io/projected/fafcfc94-94c1-45ee-a507-c26613679560-kube-api-access-p84b7\") pod \"payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf\" (UID: \"fafcfc94-94c1-45ee-a507-c26613679560\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:44:07.936751 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:07.936695    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjclp\" (UniqueName: \"kubernetes.io/projected/a1f2c109-17c9-4045-8e75-aaf57109caf1-kube-api-access-bjclp\") pod \"payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22\" (UID: \"a1f2c109-17c9-4045-8e75-aaf57109caf1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:44:08.037848 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.037815    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1f2c109-17c9-4045-8e75-aaf57109caf1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22\" (UID: \"a1f2c109-17c9-4045-8e75-aaf57109caf1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:44:08.038021 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.037891    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p84b7\" (UniqueName: \"kubernetes.io/projected/fafcfc94-94c1-45ee-a507-c26613679560-kube-api-access-p84b7\") pod \"payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf\" (UID: \"fafcfc94-94c1-45ee-a507-c26613679560\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:44:08.038021 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.037941    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjclp\" (UniqueName: \"kubernetes.io/projected/a1f2c109-17c9-4045-8e75-aaf57109caf1-kube-api-access-bjclp\") pod \"payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22\" (UID: \"a1f2c109-17c9-4045-8e75-aaf57109caf1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:44:08.038021 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.037997    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fafcfc94-94c1-45ee-a507-c26613679560-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf\" (UID: \"fafcfc94-94c1-45ee-a507-c26613679560\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:44:08.038589 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.038570    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1f2c109-17c9-4045-8e75-aaf57109caf1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22\" (UID: \"a1f2c109-17c9-4045-8e75-aaf57109caf1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:44:08.038675 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.038660    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fafcfc94-94c1-45ee-a507-c26613679560-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf\" (UID: \"fafcfc94-94c1-45ee-a507-c26613679560\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:44:08.046951 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.046923    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p84b7\" (UniqueName: \"kubernetes.io/projected/fafcfc94-94c1-45ee-a507-c26613679560-kube-api-access-p84b7\") pod \"payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf\" (UID: \"fafcfc94-94c1-45ee-a507-c26613679560\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:44:08.047094 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.047075    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjclp\" (UniqueName: \"kubernetes.io/projected/a1f2c109-17c9-4045-8e75-aaf57109caf1-kube-api-access-bjclp\") pod \"payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22\" (UID: \"a1f2c109-17c9-4045-8e75-aaf57109caf1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:44:08.189235 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.189160    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:44:08.227754 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.227718    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:44:08.468423 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.468402    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"]
Sep 15 12:44:08.471188 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:44:08.471153    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1f2c109_17c9_4045_8e75_aaf57109caf1.slice/crio-fe6e11b9f3d0c7c2bf907eb0f3460da80e85db18bc647c20e1592c4a60f6b3de WatchSource:0}: Error finding container fe6e11b9f3d0c7c2bf907eb0f3460da80e85db18bc647c20e1592c4a60f6b3de: Status 404 returned error can't find the container with id fe6e11b9f3d0c7c2bf907eb0f3460da80e85db18bc647c20e1592c4a60f6b3de
Sep 15 12:44:08.507251 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:08.507231    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"]
Sep 15 12:44:08.512792 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:44:08.512755    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfafcfc94_94c1_45ee_a507_c26613679560.slice/crio-09041ba69396b848b8f3c121c0ed3828416b759f19a0d506116ce6d11cbb49e2 WatchSource:0}: Error finding container 09041ba69396b848b8f3c121c0ed3828416b759f19a0d506116ce6d11cbb49e2: Status 404 returned error can't find the container with id 09041ba69396b848b8f3c121c0ed3828416b759f19a0d506116ce6d11cbb49e2
Sep 15 12:44:09.450579 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:09.450542    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf" event={"ID":"fafcfc94-94c1-45ee-a507-c26613679560","Type":"ContainerStarted","Data":"e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1"}
Sep 15 12:44:09.451032 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:09.450599    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf" event={"ID":"fafcfc94-94c1-45ee-a507-c26613679560","Type":"ContainerStarted","Data":"09041ba69396b848b8f3c121c0ed3828416b759f19a0d506116ce6d11cbb49e2"}
Sep 15 12:44:09.451032 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:09.450627    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:44:09.452233 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:09.452207    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22" event={"ID":"a1f2c109-17c9-4045-8e75-aaf57109caf1","Type":"ContainerStarted","Data":"7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191"}
Sep 15 12:44:09.452354 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:09.452256    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22" event={"ID":"a1f2c109-17c9-4045-8e75-aaf57109caf1","Type":"ContainerStarted","Data":"fe6e11b9f3d0c7c2bf907eb0f3460da80e85db18bc647c20e1592c4a60f6b3de"}
Sep 15 12:44:09.452354 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:09.452289    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:44:09.466716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:09.466674    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf" podStartSLOduration=2.466661545 podStartE2EDuration="2.466661545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:44:09.465201798 +0000 UTC m=+1117.277113252" watchObservedRunningTime="2026-09-15 12:44:09.466661545 +0000 UTC m=+1117.278572988"
Sep 15 12:44:09.481765 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:09.481723    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22" podStartSLOduration=2.481705533 podStartE2EDuration="2.481705533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:44:09.481371121 +0000 UTC m=+1117.293282563" watchObservedRunningTime="2026-09-15 12:44:09.481705533 +0000 UTC m=+1117.293616976"
Sep 15 12:44:18.105915 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:18.105883    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"]
Sep 15 12:44:18.106375 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:18.106168    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy" containerID="cri-o://99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137" gracePeriod=30
Sep 15 12:44:20.460002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:20.459966    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:44:20.460484 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:20.460210    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:44:23.359223 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.359198    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:44:23.500953 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.500922    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-podinfo\") pod \"264ed769-9373-42c3-a2b4-01b26ebb2a82\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") "
Sep 15 12:44:23.501133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.500985    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsgjm\" (UniqueName: \"kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-kube-api-access-bsgjm\") pod \"264ed769-9373-42c3-a2b4-01b26ebb2a82\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") "
Sep 15 12:44:23.501133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501017    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-token\") pod \"264ed769-9373-42c3-a2b4-01b26ebb2a82\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") "
Sep 15 12:44:23.501133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501053    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-data\") pod \"264ed769-9373-42c3-a2b4-01b26ebb2a82\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") "
Sep 15 12:44:23.501133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501107    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-credential-socket\") pod \"264ed769-9373-42c3-a2b4-01b26ebb2a82\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") "
Sep 15 12:44:23.501358 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501146    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-socket\") pod \"264ed769-9373-42c3-a2b4-01b26ebb2a82\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") "
Sep 15 12:44:23.501358 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501193    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/264ed769-9373-42c3-a2b4-01b26ebb2a82-istiod-ca-cert\") pod \"264ed769-9373-42c3-a2b4-01b26ebb2a82\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") "
Sep 15 12:44:23.501358 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501236    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-certs\") pod \"264ed769-9373-42c3-a2b4-01b26ebb2a82\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") "
Sep 15 12:44:23.501358 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501291    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-envoy\") pod \"264ed769-9373-42c3-a2b4-01b26ebb2a82\" (UID: \"264ed769-9373-42c3-a2b4-01b26ebb2a82\") "
Sep 15 12:44:23.501559 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501410    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-data" (OuterVolumeSpecName: "istio-data") pod "264ed769-9373-42c3-a2b4-01b26ebb2a82" (UID: "264ed769-9373-42c3-a2b4-01b26ebb2a82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:44:23.501617 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501584    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "264ed769-9373-42c3-a2b4-01b26ebb2a82" (UID: "264ed769-9373-42c3-a2b4-01b26ebb2a82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:44:23.501819 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501724    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:44:23.501819 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501747    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:44:23.501981 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501863    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "264ed769-9373-42c3-a2b4-01b26ebb2a82" (UID: "264ed769-9373-42c3-a2b4-01b26ebb2a82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:44:23.501981 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501888    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/264ed769-9373-42c3-a2b4-01b26ebb2a82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "264ed769-9373-42c3-a2b4-01b26ebb2a82" (UID: "264ed769-9373-42c3-a2b4-01b26ebb2a82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:44:23.502095 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.501975    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "264ed769-9373-42c3-a2b4-01b26ebb2a82" (UID: "264ed769-9373-42c3-a2b4-01b26ebb2a82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:44:23.503528 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.503507    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "264ed769-9373-42c3-a2b4-01b26ebb2a82" (UID: "264ed769-9373-42c3-a2b4-01b26ebb2a82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:44:23.503528 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.503525    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-kube-api-access-bsgjm" (OuterVolumeSpecName: "kube-api-access-bsgjm") pod "264ed769-9373-42c3-a2b4-01b26ebb2a82" (UID: "264ed769-9373-42c3-a2b4-01b26ebb2a82"). InnerVolumeSpecName "kube-api-access-bsgjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:44:23.504004 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.503604    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-token" (OuterVolumeSpecName: "istio-token") pod "264ed769-9373-42c3-a2b4-01b26ebb2a82" (UID: "264ed769-9373-42c3-a2b4-01b26ebb2a82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:44:23.504242 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.504219    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "264ed769-9373-42c3-a2b4-01b26ebb2a82" (UID: "264ed769-9373-42c3-a2b4-01b26ebb2a82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:44:23.535800 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.535751    2586 generic.go:356] "Generic (PLEG): container finished" podID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerID="99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137" exitCode=0
Sep 15 12:44:23.535907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.535822    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"
Sep 15 12:44:23.535907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.535827    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" event={"ID":"264ed769-9373-42c3-a2b4-01b26ebb2a82","Type":"ContainerDied","Data":"99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137"}
Sep 15 12:44:23.535907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.535901    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t" event={"ID":"264ed769-9373-42c3-a2b4-01b26ebb2a82","Type":"ContainerDied","Data":"8b060d37cb35bb31de7c118cdca3d1aa49f8e5b6393d63170d0c8e1be53ed2ad"}
Sep 15 12:44:23.536024 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.535929    2586 scope.go:117] "RemoveContainer" containerID="99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137"
Sep 15 12:44:23.547214 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.547192    2586 scope.go:117] "RemoveContainer" containerID="10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937"
Sep 15 12:44:23.556872 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.556852    2586 scope.go:117] "RemoveContainer" containerID="99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137"
Sep 15 12:44:23.557161 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:44:23.557141    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137\": container with ID starting with 99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137 not found: ID does not exist" containerID="99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137"
Sep 15 12:44:23.557230 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.557170    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137"} err="failed to get container status \"99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137\": rpc error: code = NotFound desc = could not find container \"99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137\": container with ID starting with 99903b0705662a4cf23f42d5f047826aff875bfcf43efd44d0c4bab45f880137 not found: ID does not exist"
Sep 15 12:44:23.557230 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.557186    2586 scope.go:117] "RemoveContainer" containerID="10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937"
Sep 15 12:44:23.557481 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:44:23.557456    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937\": container with ID starting with 10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937 not found: ID does not exist" containerID="10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937"
Sep 15 12:44:23.557537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.557490    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937"} err="failed to get container status \"10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937\": rpc error: code = NotFound desc = could not find container \"10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937\": container with ID starting with 10be6afb766da4714c9e9f93d3b2da95820ca83940ab9ccf8cd1b1bef786a937 not found: ID does not exist"
Sep 15 12:44:23.564582 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.564518    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"]
Sep 15 12:44:23.567739 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.567715    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-0f955e-openshift-default-7dc6656fbd-47s5t"]
Sep 15 12:44:23.602650 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.602624    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsgjm\" (UniqueName: \"kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-kube-api-access-bsgjm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:44:23.602743 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.602653    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:44:23.602743 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.602669    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:44:23.602743 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.602682    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/264ed769-9373-42c3-a2b4-01b26ebb2a82-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:44:23.602743 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.602696    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:44:23.602743 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.602707    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:44:23.602743 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:23.602719    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/264ed769-9373-42c3-a2b4-01b26ebb2a82-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:44:24.652877 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:24.652847    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" path="/var/lib/kubelet/pods/264ed769-9373-42c3-a2b4-01b26ebb2a82/volumes"
Sep 15 12:44:34.629640 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.629599    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"]
Sep 15 12:44:34.630248 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.630229    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy"
Sep 15 12:44:34.630321 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.630251    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy"
Sep 15 12:44:34.630374 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.630339    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy"
Sep 15 12:44:34.630374 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.630350    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy"
Sep 15 12:44:34.635110 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.635087    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.637517 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.637469    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-d9e7a1-openshift-default-dockercfg-7rhm8\""
Sep 15 12:44:34.645378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.645350    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"]
Sep 15 12:44:34.811497 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.811454    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a424e6d-cf84-4965-8142-1998d0fd624a-istiod-ca-cert\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.811702 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.811574    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jgfq\" (UniqueName: \"kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-kube-api-access-9jgfq\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.811702 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.811626    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-socket\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.811702 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.811645    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-envoy\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.811702 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.811668    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-token\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.811902 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.811755    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-podinfo\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.811902 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.811846    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-credential-socket\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.811902 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.811883    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-data\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.811902 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.811901    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-certs\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913240 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913130    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a424e6d-cf84-4965-8142-1998d0fd624a-istiod-ca-cert\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913240 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913217    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jgfq\" (UniqueName: \"kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-kube-api-access-9jgfq\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913486 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913247    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-socket\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913486 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913270    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-envoy\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913486 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913301    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-token\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913486 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913345    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-podinfo\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913486 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913401    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-credential-socket\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913486 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913450    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-data\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913486 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913477    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-certs\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913961 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913692    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-socket\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913961 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913758    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-credential-socket\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913961 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913867    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-certs\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.913961 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.913894    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-data\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.914152 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.914053    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a424e6d-cf84-4965-8142-1998d0fd624a-istiod-ca-cert\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.915860 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.915839    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-envoy\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.916081 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.916064    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-podinfo\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.920745 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.920720    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-token\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.920913 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.920896    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jgfq\" (UniqueName: \"kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-kube-api-access-9jgfq\") pod \"e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") " pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:34.948002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:34.947971    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:35.231458 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:35.231433    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"]
Sep 15 12:44:35.233185 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:44:35.233154    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a424e6d_cf84_4965_8142_1998d0fd624a.slice/crio-abc45e976edf43c1673c63f25718538375c028fa387cc22f0d5e444a69bfae6a WatchSource:0}: Error finding container abc45e976edf43c1673c63f25718538375c028fa387cc22f0d5e444a69bfae6a: Status 404 returned error can't find the container with id abc45e976edf43c1673c63f25718538375c028fa387cc22f0d5e444a69bfae6a
Sep 15 12:44:35.235737 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:35.235705    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:44:35.235819 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:35.235794    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:44:35.235868 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:35.235832    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:44:35.605582 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:35.605548    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" event={"ID":"3a424e6d-cf84-4965-8142-1998d0fd624a","Type":"ContainerStarted","Data":"a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38"}
Sep 15 12:44:35.605818 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:35.605600    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" event={"ID":"3a424e6d-cf84-4965-8142-1998d0fd624a","Type":"ContainerStarted","Data":"abc45e976edf43c1673c63f25718538375c028fa387cc22f0d5e444a69bfae6a"}
Sep 15 12:44:35.628725 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:35.628673    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" podStartSLOduration=1.628654755 podStartE2EDuration="1.628654755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:44:35.626474075 +0000 UTC m=+1143.438385531" watchObservedRunningTime="2026-09-15 12:44:35.628654755 +0000 UTC m=+1143.440566197"
Sep 15 12:44:35.948474 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:35.948359    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:36.954657 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:36.954627    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:37.619222 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:37.619189    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:37.620109 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:37.620083    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:44:41.685964 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.685927    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"]
Sep 15 12:44:41.686673 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.686651    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy"
Sep 15 12:44:41.686731 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.686678    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="264ed769-9373-42c3-a2b4-01b26ebb2a82" containerName="istio-proxy"
Sep 15 12:44:41.690663 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.690640    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.693282 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.693260    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-d9e7a1-serving-cert\""
Sep 15 12:44:41.701567 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.701304    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"]
Sep 15 12:44:41.787065 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.787027    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.787273 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.787129    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fr2d\" (UniqueName: \"kubernetes.io/projected/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-kube-api-access-8fr2d\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.787273 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.787168    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.787273 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.787211    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.887735 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.887696    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fr2d\" (UniqueName: \"kubernetes.io/projected/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-kube-api-access-8fr2d\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.887986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.887843    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.887986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.887881    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.887986 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.887950    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.888160 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:44:41.887983    2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-serving-cert: secret "maas-api-e2e-shared-b-w2-d9e7a1-serving-cert" not found
Sep 15 12:44:41.888160 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:44:41.888047    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-maas-api-tls podName:eaa5c813-a6be-42dd-a3d6-2f33c78699f0 nodeName:}" failed. No retries permitted until 2026-09-15 12:44:42.388030348 +0000 UTC m=+1150.199941780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-maas-api-tls") pod "maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc" (UID: "eaa5c813-a6be-42dd-a3d6-2f33c78699f0") : secret "maas-api-e2e-shared-b-w2-d9e7a1-serving-cert" not found
Sep 15 12:44:41.888582 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.888557    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.890527 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.890508    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:41.898482 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:41.898451    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fr2d\" (UniqueName: \"kubernetes.io/projected/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-kube-api-access-8fr2d\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:42.392053 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:42.392017    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:42.394566 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:42.394538    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:42.606661 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:42.606614    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:42.895689 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:42.895572    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"]
Sep 15 12:44:42.898085 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:44:42.898049    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa5c813_a6be_42dd_a3d6_2f33c78699f0.slice/crio-a759e299c9bcb98bcfa8e58c6158e72b07b49395dae75de2df7af0ab3039a549 WatchSource:0}: Error finding container a759e299c9bcb98bcfa8e58c6158e72b07b49395dae75de2df7af0ab3039a549: Status 404 returned error can't find the container with id a759e299c9bcb98bcfa8e58c6158e72b07b49395dae75de2df7af0ab3039a549
Sep 15 12:44:43.656897 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:43.656866    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc" event={"ID":"eaa5c813-a6be-42dd-a3d6-2f33c78699f0","Type":"ContainerStarted","Data":"a759e299c9bcb98bcfa8e58c6158e72b07b49395dae75de2df7af0ab3039a549"}
Sep 15 12:44:44.663106 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:44.663060    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc" event={"ID":"eaa5c813-a6be-42dd-a3d6-2f33c78699f0","Type":"ContainerStarted","Data":"77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b"}
Sep 15 12:44:44.663524 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:44.663191    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:44:44.681240 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:44.681195    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc" podStartSLOduration=2.918994534 podStartE2EDuration="3.68117919s" totalImagesPullingTime="762.184656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:44:44.679808735 +0000 UTC m=+1152.491720179" watchObservedRunningTime="2026-09-15 12:44:44.68117919 +0000 UTC m=+1152.493090633"
Sep 15 12:44:50.674819 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:44:50.674766    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:45:00.771214 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:00.771177    2586 generic.go:356] "Generic (PLEG): container finished" podID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerID="a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38" exitCode=0
Sep 15 12:45:00.771630 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:00.771233    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" event={"ID":"3a424e6d-cf84-4965-8142-1998d0fd624a","Type":"ContainerDied","Data":"a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38"}
Sep 15 12:45:00.771707 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:00.771684    2586 scope.go:117] "RemoveContainer" containerID="a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38"
Sep 15 12:45:00.772915 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:00.772882    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:45:00.772979 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:00.772960    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:45:00.773027 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:00.772989    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:45:01.778369 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:01.778324    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" event={"ID":"3a424e6d-cf84-4965-8142-1998d0fd624a","Type":"ContainerStarted","Data":"eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117"}
Sep 15 12:45:01.948703 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:01.948662    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:45:01.948994 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:01.948970    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 15 12:45:01.949055 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:01.949034    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 15 12:45:02.948943 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:02.948907    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 15 12:45:02.949364 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:02.948964    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 15 12:45:03.949299 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:03.949261    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 15 12:45:03.949686 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:03.949325    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 15 12:45:04.949091 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:04.949052    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 15 12:45:04.949272 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:04.949112    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 15 12:45:05.948414 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:05.948374    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 15 12:45:05.948867 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:05.948436    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 15 12:45:06.948724 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:06.948682    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 15 12:45:06.949184 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:06.948753    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 15 12:45:07.619507 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:07.619466    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:45:07.955346 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:07.955260    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:45:07.956618 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:07.956590    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:45:10.751090 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.751045    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"]
Sep 15 12:45:10.754025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.754002    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.757327 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.757289    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-37b6a5\"/\"openshift-service-ca.crt\""
Sep 15 12:45:10.757327 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.757297    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-37b6a5\"/\"noten-model-37b6a5-kserve-self-signed-certs\""
Sep 15 12:45:10.757544 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.757334    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-37b6a5\"/\"default-dockercfg-bsq2x\""
Sep 15 12:45:10.757544 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.757344    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-37b6a5\"/\"kube-root-ca.crt\""
Sep 15 12:45:10.766019 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.765993    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"]
Sep 15 12:45:10.813000 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.812961    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-model-cache\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.813206 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.813007    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-home\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.813206 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.813042    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42bce67-81db-43f9-83d6-51160a724191-tls-certs\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.813206 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.813159    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5qrk\" (UniqueName: \"kubernetes.io/projected/b42bce67-81db-43f9-83d6-51160a724191-kube-api-access-g5qrk\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.813206 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.813194    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-tmp-dir\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.813363 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.813226    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-dshm\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.914100 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.914058    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-tmp-dir\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.914100 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.914104    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-dshm\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.914313 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.914164    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-model-cache\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.914313 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.914184    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-home\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.914411 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.914307    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42bce67-81db-43f9-83d6-51160a724191-tls-certs\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.914500 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.914476    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5qrk\" (UniqueName: \"kubernetes.io/projected/b42bce67-81db-43f9-83d6-51160a724191-kube-api-access-g5qrk\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.914576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.914497    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-tmp-dir\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.914576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.914532    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-model-cache\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.914661 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.914606    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-home\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.916549 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.916524    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-dshm\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.916700 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.916682    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42bce67-81db-43f9-83d6-51160a724191-tls-certs\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:10.922128 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:10.922097    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5qrk\" (UniqueName: \"kubernetes.io/projected/b42bce67-81db-43f9-83d6-51160a724191-kube-api-access-g5qrk\") pod \"noten-model-37b6a5-kserve-6f7d74cff9-skxsv\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") " pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:11.065614 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:11.065499    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:11.338341 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:11.338311    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"]
Sep 15 12:45:11.340535 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:45:11.340502    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb42bce67_81db_43f9_83d6_51160a724191.slice/crio-1b61c7d7fb7d559fd60ea60f98940371aa25c87bd3ca74e0f15591d72c6394cb WatchSource:0}: Error finding container 1b61c7d7fb7d559fd60ea60f98940371aa25c87bd3ca74e0f15591d72c6394cb: Status 404 returned error can't find the container with id 1b61c7d7fb7d559fd60ea60f98940371aa25c87bd3ca74e0f15591d72c6394cb
Sep 15 12:45:11.841497 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:11.841462    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv" event={"ID":"b42bce67-81db-43f9-83d6-51160a724191","Type":"ContainerStarted","Data":"2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f"}
Sep 15 12:45:11.841497 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:11.841504    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv" event={"ID":"b42bce67-81db-43f9-83d6-51160a724191","Type":"ContainerStarted","Data":"1b61c7d7fb7d559fd60ea60f98940371aa25c87bd3ca74e0f15591d72c6394cb"}
Sep 15 12:45:11.862024 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:11.861966    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv" podStartSLOduration=1.861947458 podStartE2EDuration="1.861947458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:45:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:11.859807466 +0000 UTC m=+1179.671718906" watchObservedRunningTime="2026-09-15 12:45:11.861947458 +0000 UTC m=+1179.673858932"
Sep 15 12:45:17.372449 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.372389    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"]
Sep 15 12:45:17.378070 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.378031    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.381156 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.380713    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-88615e8e-openshift-default-dockercfg-9vtzv\""
Sep 15 12:45:17.388315 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.388280    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"]
Sep 15 12:45:17.498803 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.498590    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-podinfo\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.498803 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.498745    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-envoy\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.499181 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.499015    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqtzf\" (UniqueName: \"kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-kube-api-access-mqtzf\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.499181 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.499121    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-token\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.499292 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.499196    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-credential-socket\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.499292 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.499229    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-socket\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.499292 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.499273    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istiod-ca-cert\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.499441 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.499317    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-data\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.499441 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.499390    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-certs\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.600625 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.600571    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-token\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.600874 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.600802    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-credential-socket\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.601250 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.601214    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-socket\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.601411 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.601282    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-credential-socket\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.601411 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.601295    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istiod-ca-cert\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.601411 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.601357    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-data\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.601411 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.601360    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-socket\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.601633 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.601417    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-certs\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.601633 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.601469    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-podinfo\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.601633 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.601511    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-envoy\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.601823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.601641    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqtzf\" (UniqueName: \"kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-kube-api-access-mqtzf\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.601823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.601672    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-data\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.602173 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.602151    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istiod-ca-cert\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.602255 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.602198    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-certs\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.604554 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.604519    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-podinfo\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.604716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.604584    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-envoy\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.609440 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.609412    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-token\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.609571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.609543    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqtzf\" (UniqueName: \"kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-kube-api-access-mqtzf\") pod \"e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") " pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.696693 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.696560    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:17.981282 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.981251    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"]
Sep 15 12:45:17.984116 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:45:17.984069    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cdaff92_1faf_4d94_9790_8b7c79511ce3.slice/crio-ff5d4b04106b94e2418120c8be24976aa71bfaaaaec93931b8fab3e1bf847d5b WatchSource:0}: Error finding container ff5d4b04106b94e2418120c8be24976aa71bfaaaaec93931b8fab3e1bf847d5b: Status 404 returned error can't find the container with id ff5d4b04106b94e2418120c8be24976aa71bfaaaaec93931b8fab3e1bf847d5b
Sep 15 12:45:17.987222 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.987171    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:45:17.987336 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.987277    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:45:17.987336 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:17.987328    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:45:18.891578 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:18.891527    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" event={"ID":"2cdaff92-1faf-4d94-9790-8b7c79511ce3","Type":"ContainerStarted","Data":"e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c"}
Sep 15 12:45:18.892074 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:18.891606    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" event={"ID":"2cdaff92-1faf-4d94-9790-8b7c79511ce3","Type":"ContainerStarted","Data":"ff5d4b04106b94e2418120c8be24976aa71bfaaaaec93931b8fab3e1bf847d5b"}
Sep 15 12:45:18.912284 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:18.912219    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" podStartSLOduration=1.912197688 podStartE2EDuration="1.912197688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:18.911169066 +0000 UTC m=+1186.723080507" watchObservedRunningTime="2026-09-15 12:45:18.912197688 +0000 UTC m=+1186.724109132"
Sep 15 12:45:19.697011 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:19.696955    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:19.703987 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:19.703951    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:19.901353 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:19.901314    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:19.902492 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:19.902448    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:21.066346 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:21.066298    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:21.066995 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:21.066382    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:21.079646 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:21.079617    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:21.926658 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:21.926614    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:45:48.074685 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:48.074646    2586 generic.go:356] "Generic (PLEG): container finished" podID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerID="e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c" exitCode=0
Sep 15 12:45:48.075172 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:48.074719    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" event={"ID":"2cdaff92-1faf-4d94-9790-8b7c79511ce3","Type":"ContainerDied","Data":"e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c"}
Sep 15 12:45:48.075224 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:48.075210    2586 scope.go:117] "RemoveContainer" containerID="e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c"
Sep 15 12:45:48.076472 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:48.076443    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:45:48.076549 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:48.076518    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:45:48.076590 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:48.076553    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:45:48.677267 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:48.677231    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"]
Sep 15 12:45:49.082459 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:49.082419    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" event={"ID":"2cdaff92-1faf-4d94-9790-8b7c79511ce3","Type":"ContainerStarted","Data":"e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d"}
Sep 15 12:45:49.698115 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:49.698074    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:49.698426 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:49.698384    2586 patch_prober.go:28] interesting pod/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 15 12:45:49.698533 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:49.698477    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 15 12:45:49.901252 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:49.901211    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:50.697336 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:50.697279    2586 patch_prober.go:28] interesting pod/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 15 12:45:50.697912 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:50.697362    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 15 12:45:51.698169 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:51.697687    2586 patch_prober.go:28] interesting pod/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 15 12:45:51.698169 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:51.697753    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 15 12:45:52.697640 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:52.697595    2586 patch_prober.go:28] interesting pod/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 15 12:45:52.697874 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:52.697666    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 15 12:45:53.697922 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:53.697872    2586 patch_prober.go:28] interesting pod/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 15 12:45:53.698484 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:53.697947    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 15 12:45:54.701835 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:54.701800    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:45:54.703056 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:45:54.703031    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:46:03.716097 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:03.716053    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv" podUID="b42bce67-81db-43f9-83d6-51160a724191" containerName="main" containerID="cri-o://2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f" gracePeriod=45
Sep 15 12:46:03.977724 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:03.977664    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:46:04.057082 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057051    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-home\") pod \"b42bce67-81db-43f9-83d6-51160a724191\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") "
Sep 15 12:46:04.057283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057109    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-tmp-dir\") pod \"b42bce67-81db-43f9-83d6-51160a724191\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") "
Sep 15 12:46:04.057283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057155    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-dshm\") pod \"b42bce67-81db-43f9-83d6-51160a724191\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") "
Sep 15 12:46:04.057283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057203    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-model-cache\") pod \"b42bce67-81db-43f9-83d6-51160a724191\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") "
Sep 15 12:46:04.057283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057261    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5qrk\" (UniqueName: \"kubernetes.io/projected/b42bce67-81db-43f9-83d6-51160a724191-kube-api-access-g5qrk\") pod \"b42bce67-81db-43f9-83d6-51160a724191\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") "
Sep 15 12:46:04.057520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057287    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42bce67-81db-43f9-83d6-51160a724191-tls-certs\") pod \"b42bce67-81db-43f9-83d6-51160a724191\" (UID: \"b42bce67-81db-43f9-83d6-51160a724191\") "
Sep 15 12:46:04.057520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057363    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-home" (OuterVolumeSpecName: "home") pod "b42bce67-81db-43f9-83d6-51160a724191" (UID: "b42bce67-81db-43f9-83d6-51160a724191"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:04.057520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057472    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b42bce67-81db-43f9-83d6-51160a724191" (UID: "b42bce67-81db-43f9-83d6-51160a724191"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:04.057520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057502    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-model-cache" (OuterVolumeSpecName: "model-cache") pod "b42bce67-81db-43f9-83d6-51160a724191" (UID: "b42bce67-81db-43f9-83d6-51160a724191"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:04.057742 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057724    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-home\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:04.057828 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057747    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-tmp-dir\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:04.057828 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.057761    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-model-cache\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:04.059463 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.059437    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-dshm" (OuterVolumeSpecName: "dshm") pod "b42bce67-81db-43f9-83d6-51160a724191" (UID: "b42bce67-81db-43f9-83d6-51160a724191"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:04.059588 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.059526    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b42bce67-81db-43f9-83d6-51160a724191-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b42bce67-81db-43f9-83d6-51160a724191" (UID: "b42bce67-81db-43f9-83d6-51160a724191"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:46:04.059588 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.059562    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b42bce67-81db-43f9-83d6-51160a724191-kube-api-access-g5qrk" (OuterVolumeSpecName: "kube-api-access-g5qrk") pod "b42bce67-81db-43f9-83d6-51160a724191" (UID: "b42bce67-81db-43f9-83d6-51160a724191"). InnerVolumeSpecName "kube-api-access-g5qrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:46:04.159023 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.158991    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42bce67-81db-43f9-83d6-51160a724191-dshm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:04.159023 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.159017    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5qrk\" (UniqueName: \"kubernetes.io/projected/b42bce67-81db-43f9-83d6-51160a724191-kube-api-access-g5qrk\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:04.159023 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.159028    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42bce67-81db-43f9-83d6-51160a724191-tls-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:04.177795 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.177752    2586 generic.go:356] "Generic (PLEG): container finished" podID="b42bce67-81db-43f9-83d6-51160a724191" containerID="2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f" exitCode=0
Sep 15 12:46:04.177993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.177828    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"
Sep 15 12:46:04.177993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.177831    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv" event={"ID":"b42bce67-81db-43f9-83d6-51160a724191","Type":"ContainerDied","Data":"2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f"}
Sep 15 12:46:04.177993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.177943    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv" event={"ID":"b42bce67-81db-43f9-83d6-51160a724191","Type":"ContainerDied","Data":"1b61c7d7fb7d559fd60ea60f98940371aa25c87bd3ca74e0f15591d72c6394cb"}
Sep 15 12:46:04.177993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.177964    2586 scope.go:117] "RemoveContainer" containerID="2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f"
Sep 15 12:46:04.187756 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.187738    2586 scope.go:117] "RemoveContainer" containerID="2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f"
Sep 15 12:46:04.188054 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:46:04.188037    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f\": container with ID starting with 2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f not found: ID does not exist" containerID="2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f"
Sep 15 12:46:04.188110 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.188063    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f"} err="failed to get container status \"2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f\": rpc error: code = NotFound desc = could not find container \"2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f\": container with ID starting with 2c29c060da5050588e0799dc3bbf60361dbc5612b3c8a4fcbef97e9204ee879f not found: ID does not exist"
Sep 15 12:46:04.200528 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.200499    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"]
Sep 15 12:46:04.202867 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.202843    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-37b6a5/noten-model-37b6a5-kserve-6f7d74cff9-skxsv"]
Sep 15 12:46:04.654448 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:04.654409    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b42bce67-81db-43f9-83d6-51160a724191" path="/var/lib/kubelet/pods/b42bce67-81db-43f9-83d6-51160a724191/volumes"
Sep 15 12:46:08.165432 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:08.165392    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"]
Sep 15 12:46:18.779309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:18.779266    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"]
Sep 15 12:46:18.779958 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:18.779839    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm" podUID="802ad009-5cec-4276-a9f5-8789c6dec747" containerName="maas-api" containerID="cri-o://f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613" gracePeriod=30
Sep 15 12:46:19.059345 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.059321    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:46:19.233425 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.233379    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bp7q\" (UniqueName: \"kubernetes.io/projected/802ad009-5cec-4276-a9f5-8789c6dec747-kube-api-access-4bp7q\") pod \"802ad009-5cec-4276-a9f5-8789c6dec747\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") "
Sep 15 12:46:19.233602 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.233484    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-metrics-certs\") pod \"802ad009-5cec-4276-a9f5-8789c6dec747\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") "
Sep 15 12:46:19.233602 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.233537    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/802ad009-5cec-4276-a9f5-8789c6dec747-openshift-service-ca\") pod \"802ad009-5cec-4276-a9f5-8789c6dec747\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") "
Sep 15 12:46:19.233695 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.233605    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-maas-api-tls\") pod \"802ad009-5cec-4276-a9f5-8789c6dec747\" (UID: \"802ad009-5cec-4276-a9f5-8789c6dec747\") "
Sep 15 12:46:19.233980 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.233946    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/802ad009-5cec-4276-a9f5-8789c6dec747-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "802ad009-5cec-4276-a9f5-8789c6dec747" (UID: "802ad009-5cec-4276-a9f5-8789c6dec747"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:46:19.235741 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.235716    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "802ad009-5cec-4276-a9f5-8789c6dec747" (UID: "802ad009-5cec-4276-a9f5-8789c6dec747"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:46:19.235876 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.235766    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "802ad009-5cec-4276-a9f5-8789c6dec747" (UID: "802ad009-5cec-4276-a9f5-8789c6dec747"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:46:19.236001 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.235983    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/802ad009-5cec-4276-a9f5-8789c6dec747-kube-api-access-4bp7q" (OuterVolumeSpecName: "kube-api-access-4bp7q") pod "802ad009-5cec-4276-a9f5-8789c6dec747" (UID: "802ad009-5cec-4276-a9f5-8789c6dec747"). InnerVolumeSpecName "kube-api-access-4bp7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:46:19.265622 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.265596    2586 generic.go:356] "Generic (PLEG): container finished" podID="802ad009-5cec-4276-a9f5-8789c6dec747" containerID="f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613" exitCode=0
Sep 15 12:46:19.265770 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.265659    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"
Sep 15 12:46:19.265770 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.265682    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm" event={"ID":"802ad009-5cec-4276-a9f5-8789c6dec747","Type":"ContainerDied","Data":"f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613"}
Sep 15 12:46:19.265770 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.265765    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm" event={"ID":"802ad009-5cec-4276-a9f5-8789c6dec747","Type":"ContainerDied","Data":"990b9ae5fd6899415c1d8237d5b900b295006c8f5711a8a983b986c62f15c2e2"}
Sep 15 12:46:19.265902 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.265824    2586 scope.go:117] "RemoveContainer" containerID="f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613"
Sep 15 12:46:19.277707 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.277689    2586 scope.go:117] "RemoveContainer" containerID="f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613"
Sep 15 12:46:19.278033 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:46:19.278010    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613\": container with ID starting with f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613 not found: ID does not exist" containerID="f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613"
Sep 15 12:46:19.278086 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.278043    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613"} err="failed to get container status \"f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613\": rpc error: code = NotFound desc = could not find container \"f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613\": container with ID starting with f223188d84d6cf9839f5136736768675ef9c2e1a8d908a1b4bd16a67ea808613 not found: ID does not exist"
Sep 15 12:46:19.290599 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.290540    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"]
Sep 15 12:46:19.294174 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.294151    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4609ba-84cd964c98-pnxvm"]
Sep 15 12:46:19.335112 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.335084    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-maas-api-tls\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:19.335112 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.335109    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bp7q\" (UniqueName: \"kubernetes.io/projected/802ad009-5cec-4276-a9f5-8789c6dec747-kube-api-access-4bp7q\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:19.335296 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.335119    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/802ad009-5cec-4276-a9f5-8789c6dec747-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:19.335296 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:19.335129    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/802ad009-5cec-4276-a9f5-8789c6dec747-openshift-service-ca\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:20.653689 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:20.653654    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="802ad009-5cec-4276-a9f5-8789c6dec747" path="/var/lib/kubelet/pods/802ad009-5cec-4276-a9f5-8789c6dec747/volumes"
Sep 15 12:46:23.186679 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.186634    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg" podUID="06dee008-23b7-425b-bc8d-7f08ddc9c9ac" containerName="main" containerID="cri-o://45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb" gracePeriod=45
Sep 15 12:46:23.449721 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.449656    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:46:23.479494 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.479462    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tls-certs\") pod \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") "
Sep 15 12:46:23.479687 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.479509    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-model-cache\") pod \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") "
Sep 15 12:46:23.479687 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.479549    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-home\") pod \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") "
Sep 15 12:46:23.479687 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.479580    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-dshm\") pod \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") "
Sep 15 12:46:23.479687 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.479618    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z5tl\" (UniqueName: \"kubernetes.io/projected/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-kube-api-access-4z5tl\") pod \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") "
Sep 15 12:46:23.480025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.479750    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tmp-dir\") pod \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\" (UID: \"06dee008-23b7-425b-bc8d-7f08ddc9c9ac\") "
Sep 15 12:46:23.480025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.479836    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-home" (OuterVolumeSpecName: "home") pod "06dee008-23b7-425b-bc8d-7f08ddc9c9ac" (UID: "06dee008-23b7-425b-bc8d-7f08ddc9c9ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:23.480025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.479888    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "06dee008-23b7-425b-bc8d-7f08ddc9c9ac" (UID: "06dee008-23b7-425b-bc8d-7f08ddc9c9ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:23.480188 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.480068    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "06dee008-23b7-425b-bc8d-7f08ddc9c9ac" (UID: "06dee008-23b7-425b-bc8d-7f08ddc9c9ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:23.480313 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.480282    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tmp-dir\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:23.480313 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.480305    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-model-cache\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:23.480438 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.480319    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-home\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:23.482002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.481955    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06dee008-23b7-425b-bc8d-7f08ddc9c9ac" (UID: "06dee008-23b7-425b-bc8d-7f08ddc9c9ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:46:23.482139 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.482118    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-kube-api-access-4z5tl" (OuterVolumeSpecName: "kube-api-access-4z5tl") pod "06dee008-23b7-425b-bc8d-7f08ddc9c9ac" (UID: "06dee008-23b7-425b-bc8d-7f08ddc9c9ac"). InnerVolumeSpecName "kube-api-access-4z5tl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:46:23.482215 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.482144    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-dshm" (OuterVolumeSpecName: "dshm") pod "06dee008-23b7-425b-bc8d-7f08ddc9c9ac" (UID: "06dee008-23b7-425b-bc8d-7f08ddc9c9ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:23.580867 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.580827    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-tls-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:23.580867 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.580857    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-dshm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:23.580867 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:23.580872    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z5tl\" (UniqueName: \"kubernetes.io/projected/06dee008-23b7-425b-bc8d-7f08ddc9c9ac-kube-api-access-4z5tl\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:24.303639 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:24.303598    2586 generic.go:356] "Generic (PLEG): container finished" podID="06dee008-23b7-425b-bc8d-7f08ddc9c9ac" containerID="45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb" exitCode=0
Sep 15 12:46:24.304130 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:24.303664    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"
Sep 15 12:46:24.304130 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:24.303690    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg" event={"ID":"06dee008-23b7-425b-bc8d-7f08ddc9c9ac","Type":"ContainerDied","Data":"45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb"}
Sep 15 12:46:24.304130 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:24.303744    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg" event={"ID":"06dee008-23b7-425b-bc8d-7f08ddc9c9ac","Type":"ContainerDied","Data":"67cba2f358ea22f7fe0fcffa41a3972cc38a412eb10044023b0e3ea6274b6c3a"}
Sep 15 12:46:24.304130 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:24.303768    2586 scope.go:117] "RemoveContainer" containerID="45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb"
Sep 15 12:46:24.314906 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:24.314882    2586 scope.go:117] "RemoveContainer" containerID="45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb"
Sep 15 12:46:24.315228 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:46:24.315198    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb\": container with ID starting with 45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb not found: ID does not exist" containerID="45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb"
Sep 15 12:46:24.315302 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:24.315231    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb"} err="failed to get container status \"45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb\": rpc error: code = NotFound desc = could not find container \"45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb\": container with ID starting with 45bfe563821734526e25bf2dcda1b7483ad1eb4c853bf162248d90a15a9515eb not found: ID does not exist"
Sep 15 12:46:24.327369 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:24.327343    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"]
Sep 15 12:46:24.330366 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:24.330343    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2e3fef/test-model-2e3fef-kserve-84b594497f-858lg"]
Sep 15 12:46:24.653248 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:24.653167    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06dee008-23b7-425b-bc8d-7f08ddc9c9ac" path="/var/lib/kubelet/pods/06dee008-23b7-425b-bc8d-7f08ddc9c9ac/volumes"
Sep 15 12:46:25.568512 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:25.568476    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"]
Sep 15 12:46:25.568941 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:25.568763    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy" containerID="cri-o://d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1" gracePeriod=30
Sep 15 12:46:30.837511 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.837443    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:46:30.960002 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.959968    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-certs\") pod \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") "
Sep 15 12:46:30.960205 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960015    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-socket\") pod \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") "
Sep 15 12:46:30.960205 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960049    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-podinfo\") pod \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") "
Sep 15 12:46:30.960205 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960082    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-token\") pod \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") "
Sep 15 12:46:30.960205 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960159    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-credential-socket\") pod \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") "
Sep 15 12:46:30.960430 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960222    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-data\") pod \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") "
Sep 15 12:46:30.960430 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960244    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "229379c4-14a8-411f-84fb-bb5d2b7dbca6" (UID: "229379c4-14a8-411f-84fb-bb5d2b7dbca6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:30.960430 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960267    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istiod-ca-cert\") pod \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") "
Sep 15 12:46:30.960430 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960352    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckrpf\" (UniqueName: \"kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-kube-api-access-ckrpf\") pod \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") "
Sep 15 12:46:30.960430 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960389    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-envoy\") pod \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\" (UID: \"229379c4-14a8-411f-84fb-bb5d2b7dbca6\") "
Sep 15 12:46:30.960430 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960388    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "229379c4-14a8-411f-84fb-bb5d2b7dbca6" (UID: "229379c4-14a8-411f-84fb-bb5d2b7dbca6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:30.960430 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960426    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-data" (OuterVolumeSpecName: "istio-data") pod "229379c4-14a8-411f-84fb-bb5d2b7dbca6" (UID: "229379c4-14a8-411f-84fb-bb5d2b7dbca6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:30.960808 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960581    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "229379c4-14a8-411f-84fb-bb5d2b7dbca6" (UID: "229379c4-14a8-411f-84fb-bb5d2b7dbca6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:30.960808 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960737    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "229379c4-14a8-411f-84fb-bb5d2b7dbca6" (UID: "229379c4-14a8-411f-84fb-bb5d2b7dbca6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:46:30.960808 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960766    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:30.960808 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960803    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:30.961035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960816    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:30.961035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.960829    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:30.962508 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.962479    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "229379c4-14a8-411f-84fb-bb5d2b7dbca6" (UID: "229379c4-14a8-411f-84fb-bb5d2b7dbca6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:30.962644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.962538    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "229379c4-14a8-411f-84fb-bb5d2b7dbca6" (UID: "229379c4-14a8-411f-84fb-bb5d2b7dbca6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:46:30.962644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.962544    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-token" (OuterVolumeSpecName: "istio-token") pod "229379c4-14a8-411f-84fb-bb5d2b7dbca6" (UID: "229379c4-14a8-411f-84fb-bb5d2b7dbca6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:46:30.962644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:30.962633    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-kube-api-access-ckrpf" (OuterVolumeSpecName: "kube-api-access-ckrpf") pod "229379c4-14a8-411f-84fb-bb5d2b7dbca6" (UID: "229379c4-14a8-411f-84fb-bb5d2b7dbca6"). InnerVolumeSpecName "kube-api-access-ckrpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:46:31.062305 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.062268    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:31.062305 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.062297    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckrpf\" (UniqueName: \"kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-kube-api-access-ckrpf\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:31.062305 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.062310    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:31.062305 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.062319    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:31.062610 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.062327    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/229379c4-14a8-411f-84fb-bb5d2b7dbca6-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:31.347211 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.347175    2586 generic.go:356] "Generic (PLEG): container finished" podID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerID="d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1" exitCode=0
Sep 15 12:46:31.347401 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.347222    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"
Sep 15 12:46:31.347401 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.347265    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" event={"ID":"229379c4-14a8-411f-84fb-bb5d2b7dbca6","Type":"ContainerDied","Data":"d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1"}
Sep 15 12:46:31.347401 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.347347    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4" event={"ID":"229379c4-14a8-411f-84fb-bb5d2b7dbca6","Type":"ContainerDied","Data":"a11f4e9a76c4521fb3a46606b042d512cc034751807cc46fba3fc0cddf1c656c"}
Sep 15 12:46:31.347401 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.347370    2586 scope.go:117] "RemoveContainer" containerID="d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1"
Sep 15 12:46:31.357428 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.357409    2586 scope.go:117] "RemoveContainer" containerID="36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c"
Sep 15 12:46:31.366394 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.366366    2586 scope.go:117] "RemoveContainer" containerID="d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1"
Sep 15 12:46:31.367039 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:46:31.366814    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1\": container with ID starting with d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1 not found: ID does not exist" containerID="d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1"
Sep 15 12:46:31.367039 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.366853    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1"} err="failed to get container status \"d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1\": rpc error: code = NotFound desc = could not find container \"d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1\": container with ID starting with d75da35d7325f71db8d1cee9ed154a7143630b4f7575cb0aa7dff2bc7df9a7c1 not found: ID does not exist"
Sep 15 12:46:31.367039 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.366881    2586 scope.go:117] "RemoveContainer" containerID="36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c"
Sep 15 12:46:31.367237 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:46:31.367156    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c\": container with ID starting with 36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c not found: ID does not exist" containerID="36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c"
Sep 15 12:46:31.367237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.367191    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c"} err="failed to get container status \"36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c\": rpc error: code = NotFound desc = could not find container \"36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c\": container with ID starting with 36585c17e366ec3a610d0633c6a71d65154d4d15fb184b54746558266f04b19c not found: ID does not exist"
Sep 15 12:46:31.369308 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.369287    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"]
Sep 15 12:46:31.374127 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.374105    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4609ba-openshift-default-576898958-qb6w4"]
Sep 15 12:46:31.450407 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.450365    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"]
Sep 15 12:46:31.451118 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451092    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy"
Sep 15 12:46:31.451118 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451118    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy"
Sep 15 12:46:31.451309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451136    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy"
Sep 15 12:46:31.451309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451144    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy"
Sep 15 12:46:31.451309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451157    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06dee008-23b7-425b-bc8d-7f08ddc9c9ac" containerName="main"
Sep 15 12:46:31.451309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451167    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="06dee008-23b7-425b-bc8d-7f08ddc9c9ac" containerName="main"
Sep 15 12:46:31.451309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451194    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="802ad009-5cec-4276-a9f5-8789c6dec747" containerName="maas-api"
Sep 15 12:46:31.451309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451202    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="802ad009-5cec-4276-a9f5-8789c6dec747" containerName="maas-api"
Sep 15 12:46:31.451309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451218    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b42bce67-81db-43f9-83d6-51160a724191" containerName="main"
Sep 15 12:46:31.451309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451227    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b42bce67-81db-43f9-83d6-51160a724191" containerName="main"
Sep 15 12:46:31.451707 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451337    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy"
Sep 15 12:46:31.451707 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451349    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="802ad009-5cec-4276-a9f5-8789c6dec747" containerName="maas-api"
Sep 15 12:46:31.451707 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451358    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" containerName="istio-proxy"
Sep 15 12:46:31.451707 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451375    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b42bce67-81db-43f9-83d6-51160a724191" containerName="main"
Sep 15 12:46:31.451707 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.451385    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="06dee008-23b7-425b-bc8d-7f08ddc9c9ac" containerName="main"
Sep 15 12:46:31.453957 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.453931    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.456397 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.456376    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-88615e8e\"/\"kube-root-ca.crt\""
Sep 15 12:46:31.456535 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.456507    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-88615e8e\"/\"default-dockercfg-mlqdc\""
Sep 15 12:46:31.456600 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.456543    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-88615e8e\"/\"e2e-lifecycle-model-88615e8e-kserve-self-signed-certs\""
Sep 15 12:46:31.456644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.456597    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-88615e8e\"/\"openshift-service-ca.crt\""
Sep 15 12:46:31.462646 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.462621    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"]
Sep 15 12:46:31.567320 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.567289    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tmp-dir\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.567523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.567333    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-home\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.567523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.567363    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-model-cache\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.567523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.567475    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tls-certs\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.567523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.567520    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-dshm\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.567730 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.567596    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8h69\" (UniqueName: \"kubernetes.io/projected/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-kube-api-access-j8h69\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.669175 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.669081    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8h69\" (UniqueName: \"kubernetes.io/projected/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-kube-api-access-j8h69\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.669175 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.669166    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tmp-dir\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.669415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.669213    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-home\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.669415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.669329    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-model-cache\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.669415 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.669401    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tls-certs\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.669576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.669427    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-dshm\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.669576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.669542    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tmp-dir\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.669677 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.669608    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-home\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.669713 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.669671    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-model-cache\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.671679 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.671650    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-dshm\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.671903 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.671886    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tls-certs\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.677503 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.677477    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8h69\" (UniqueName: \"kubernetes.io/projected/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-kube-api-access-j8h69\") pod \"e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") " pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:31.765801 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:31.765745    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:32.039753 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:32.039723    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"]
Sep 15 12:46:32.041283 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:46:32.041257    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29e37ad2_f610_4cd0_920d_196c0fb6a4cc.slice/crio-3f086044628c67dd5f7210045feb894fec9a4a414b61ac6d1a2ad06f677d2c67 WatchSource:0}: Error finding container 3f086044628c67dd5f7210045feb894fec9a4a414b61ac6d1a2ad06f677d2c67: Status 404 returned error can't find the container with id 3f086044628c67dd5f7210045feb894fec9a4a414b61ac6d1a2ad06f677d2c67
Sep 15 12:46:32.355988 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:32.355897    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n" event={"ID":"29e37ad2-f610-4cd0-920d-196c0fb6a4cc","Type":"ContainerStarted","Data":"5e7484aafb0b27a0cf24d330e0eddeddf7e0b36aec38f8eef2b3384aa6b8578f"}
Sep 15 12:46:32.355988 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:32.355944    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n" event={"ID":"29e37ad2-f610-4cd0-920d-196c0fb6a4cc","Type":"ContainerStarted","Data":"3f086044628c67dd5f7210045feb894fec9a4a414b61ac6d1a2ad06f677d2c67"}
Sep 15 12:46:32.371420 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:32.371370    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n" podStartSLOduration=1.371342589 podStartE2EDuration="1.371342589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:46:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:46:32.370300733 +0000 UTC m=+1260.182212173" watchObservedRunningTime="2026-09-15 12:46:32.371342589 +0000 UTC m=+1260.183254030"
Sep 15 12:46:32.653657 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:32.653581    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="229379c4-14a8-411f-84fb-bb5d2b7dbca6" path="/var/lib/kubelet/pods/229379c4-14a8-411f-84fb-bb5d2b7dbca6/volumes"
Sep 15 12:46:41.766143 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:41.766100    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:41.766609 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:41.766159    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:41.778935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:41.778909    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:42.425239 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:42.425207    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:46:42.884577 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:42.884542    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"]
Sep 15 12:46:42.885070 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:42.884861    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy" containerID="cri-o://ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99" gracePeriod=30
Sep 15 12:46:47.127420 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:47.127378    2586 patch_prober.go:28] interesting pod/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 12:46:47.127941 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:47.127447    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 12:46:48.146082 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.146054    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:46:48.245702 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.245672    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-token\") pod \"bff85a67-fbc8-4f77-acf4-33305db8e41e\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") "
Sep 15 12:46:48.245895 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.245711    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-data\") pod \"bff85a67-fbc8-4f77-acf4-33305db8e41e\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") "
Sep 15 12:46:48.245895 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.245743    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-socket\") pod \"bff85a67-fbc8-4f77-acf4-33305db8e41e\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") "
Sep 15 12:46:48.245895 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.245814    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-envoy\") pod \"bff85a67-fbc8-4f77-acf4-33305db8e41e\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") "
Sep 15 12:46:48.246025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.245878    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-credential-socket\") pod \"bff85a67-fbc8-4f77-acf4-33305db8e41e\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") "
Sep 15 12:46:48.246025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.245935    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff85a67-fbc8-4f77-acf4-33305db8e41e-istiod-ca-cert\") pod \"bff85a67-fbc8-4f77-acf4-33305db8e41e\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") "
Sep 15 12:46:48.246025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.245962    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-podinfo\") pod \"bff85a67-fbc8-4f77-acf4-33305db8e41e\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") "
Sep 15 12:46:48.246025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.245994    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-certs\") pod \"bff85a67-fbc8-4f77-acf4-33305db8e41e\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") "
Sep 15 12:46:48.246025 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.245996    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-data" (OuterVolumeSpecName: "istio-data") pod "bff85a67-fbc8-4f77-acf4-33305db8e41e" (UID: "bff85a67-fbc8-4f77-acf4-33305db8e41e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:48.246277 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.246092    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl7c8\" (UniqueName: \"kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-kube-api-access-dl7c8\") pod \"bff85a67-fbc8-4f77-acf4-33305db8e41e\" (UID: \"bff85a67-fbc8-4f77-acf4-33305db8e41e\") "
Sep 15 12:46:48.246277 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.246115    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bff85a67-fbc8-4f77-acf4-33305db8e41e" (UID: "bff85a67-fbc8-4f77-acf4-33305db8e41e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:48.246277 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.246130    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bff85a67-fbc8-4f77-acf4-33305db8e41e" (UID: "bff85a67-fbc8-4f77-acf4-33305db8e41e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:48.246277 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.246251    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bff85a67-fbc8-4f77-acf4-33305db8e41e" (UID: "bff85a67-fbc8-4f77-acf4-33305db8e41e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:48.246480 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.246360    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bff85a67-fbc8-4f77-acf4-33305db8e41e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bff85a67-fbc8-4f77-acf4-33305db8e41e" (UID: "bff85a67-fbc8-4f77-acf4-33305db8e41e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:46:48.246549 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.246486    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:48.246549 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.246506    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff85a67-fbc8-4f77-acf4-33305db8e41e-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:48.246549 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.246522    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:48.246549 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.246534    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:48.246549 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.246547    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:48.248280 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.248257    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-kube-api-access-dl7c8" (OuterVolumeSpecName: "kube-api-access-dl7c8") pod "bff85a67-fbc8-4f77-acf4-33305db8e41e" (UID: "bff85a67-fbc8-4f77-acf4-33305db8e41e"). InnerVolumeSpecName "kube-api-access-dl7c8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:46:48.248392 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.248369    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bff85a67-fbc8-4f77-acf4-33305db8e41e" (UID: "bff85a67-fbc8-4f77-acf4-33305db8e41e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:46:48.248449 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.248410    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-token" (OuterVolumeSpecName: "istio-token") pod "bff85a67-fbc8-4f77-acf4-33305db8e41e" (UID: "bff85a67-fbc8-4f77-acf4-33305db8e41e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:46:48.248449 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.248414    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bff85a67-fbc8-4f77-acf4-33305db8e41e" (UID: "bff85a67-fbc8-4f77-acf4-33305db8e41e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:48.347858 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.347823    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:48.347858 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.347851    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl7c8\" (UniqueName: \"kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-kube-api-access-dl7c8\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:48.347858 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.347864    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:48.348123 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.347872    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff85a67-fbc8-4f77-acf4-33305db8e41e-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:46:48.454662 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.454626    2586 generic.go:356] "Generic (PLEG): container finished" podID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerID="ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99" exitCode=0
Sep 15 12:46:48.454892 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.454690    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"
Sep 15 12:46:48.454892 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.454713    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" event={"ID":"bff85a67-fbc8-4f77-acf4-33305db8e41e","Type":"ContainerDied","Data":"ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99"}
Sep 15 12:46:48.454892 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.454819    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn" event={"ID":"bff85a67-fbc8-4f77-acf4-33305db8e41e","Type":"ContainerDied","Data":"098e70d4f043f28604e5dd30f236414e1005ba502ea89bf99e2420eae9b1e8dd"}
Sep 15 12:46:48.454892 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.454851    2586 scope.go:117] "RemoveContainer" containerID="ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99"
Sep 15 12:46:48.464980 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.464960    2586 scope.go:117] "RemoveContainer" containerID="1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3"
Sep 15 12:46:48.473160 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.473142    2586 scope.go:117] "RemoveContainer" containerID="ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99"
Sep 15 12:46:48.473532 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:46:48.473506    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99\": container with ID starting with ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99 not found: ID does not exist" containerID="ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99"
Sep 15 12:46:48.473532 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.473541    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99"} err="failed to get container status \"ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99\": rpc error: code = NotFound desc = could not find container \"ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99\": container with ID starting with ae4da01030205e0568c0689e3089b7872c862f98454c31bae9ea955533cc2f99 not found: ID does not exist"
Sep 15 12:46:48.473771 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.473566    2586 scope.go:117] "RemoveContainer" containerID="1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3"
Sep 15 12:46:48.473975 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:46:48.473949    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3\": container with ID starting with 1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3 not found: ID does not exist" containerID="1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3"
Sep 15 12:46:48.474086 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.473982    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3"} err="failed to get container status \"1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3\": rpc error: code = NotFound desc = could not find container \"1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3\": container with ID starting with 1148eaf7dff531631a9a928c18ac26b905f45db059692d4b24e5c79291f221a3 not found: ID does not exist"
Sep 15 12:46:48.476488 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.476469    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"]
Sep 15 12:46:48.480146 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.480126    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-2e3fef-openshift-default-685c65cccd-7wbvn"]
Sep 15 12:46:48.654182 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:48.654100    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" path="/var/lib/kubelet/pods/bff85a67-fbc8-4f77-acf4-33305db8e41e/volumes"
Sep 15 12:46:50.865626 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.865591    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"]
Sep 15 12:46:50.866317 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.866292    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy"
Sep 15 12:46:50.866488 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.866471    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy"
Sep 15 12:46:50.866599 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.866587    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy"
Sep 15 12:46:50.866706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.866684    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy"
Sep 15 12:46:50.866893 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.866867    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy"
Sep 15 12:46:50.871682 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.871651    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:50.874243 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.874219    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-29a9e9-serving-cert\""
Sep 15 12:46:50.883795 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.883710    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"]
Sep 15 12:46:50.982824 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.982311    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:50.982824 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.982373    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ec550d-4112-4b55-87b0-5318a1ade216-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:50.982824 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.982435    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j9rj\" (UniqueName: \"kubernetes.io/projected/89ec550d-4112-4b55-87b0-5318a1ade216-kube-api-access-5j9rj\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:50.982824 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:50.982647    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:51.084143 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.084109    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:51.084329 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.084149    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ec550d-4112-4b55-87b0-5318a1ade216-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:51.084329 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:46:51.084259    2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-serving-cert: secret "maas-api-e2e-shared-a-w3-29a9e9-serving-cert" not found
Sep 15 12:46:51.084329 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.084273    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j9rj\" (UniqueName: \"kubernetes.io/projected/89ec550d-4112-4b55-87b0-5318a1ade216-kube-api-access-5j9rj\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:51.084469 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:46:51.084333    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-maas-api-tls podName:89ec550d-4112-4b55-87b0-5318a1ade216 nodeName:}" failed. No retries permitted until 2026-09-15 12:46:51.584314735 +0000 UTC m=+1279.396226180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-maas-api-tls") pod "maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7" (UID: "89ec550d-4112-4b55-87b0-5318a1ade216") : secret "maas-api-e2e-shared-a-w3-29a9e9-serving-cert" not found
Sep 15 12:46:51.084469 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.084414    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:51.084730 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.084713    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ec550d-4112-4b55-87b0-5318a1ade216-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:51.086884 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.086862    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:51.092286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.092263    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j9rj\" (UniqueName: \"kubernetes.io/projected/89ec550d-4112-4b55-87b0-5318a1ade216-kube-api-access-5j9rj\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:51.471380 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.471345    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"]
Sep 15 12:46:51.471952 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.471934    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="bff85a67-fbc8-4f77-acf4-33305db8e41e" containerName="istio-proxy"
Sep 15 12:46:51.475246 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.475231    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 12:46:51.477599 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.477573    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-29a9e9\""
Sep 15 12:46:51.477756 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.477740    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-29a9e9-dockercfg-nwzg7\""
Sep 15 12:46:51.487509 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.487485    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"]
Sep 15 12:46:51.589531 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.589489    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:51.590106 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.590057    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/287fc015-568a-40cf-bb2a-0f4e46470858-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs\" (UID: \"287fc015-568a-40cf-bb2a-0f4e46470858\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 12:46:51.590266 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.590153    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmb6p\" (UniqueName: \"kubernetes.io/projected/287fc015-568a-40cf-bb2a-0f4e46470858-kube-api-access-mmb6p\") pod \"payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs\" (UID: \"287fc015-568a-40cf-bb2a-0f4e46470858\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 12:46:51.592708 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.592679    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:51.691131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.691090    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/287fc015-568a-40cf-bb2a-0f4e46470858-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs\" (UID: \"287fc015-568a-40cf-bb2a-0f4e46470858\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 12:46:51.691309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.691151    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmb6p\" (UniqueName: \"kubernetes.io/projected/287fc015-568a-40cf-bb2a-0f4e46470858-kube-api-access-mmb6p\") pod \"payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs\" (UID: \"287fc015-568a-40cf-bb2a-0f4e46470858\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 12:46:51.691874 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.691857    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/287fc015-568a-40cf-bb2a-0f4e46470858-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs\" (UID: \"287fc015-568a-40cf-bb2a-0f4e46470858\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 12:46:51.699992 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.699950    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmb6p\" (UniqueName: \"kubernetes.io/projected/287fc015-568a-40cf-bb2a-0f4e46470858-kube-api-access-mmb6p\") pod \"payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs\" (UID: \"287fc015-568a-40cf-bb2a-0f4e46470858\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 12:46:51.787576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.787539    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 12:46:51.793810 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:51.793767    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:52.068490 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:52.068458    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"]
Sep 15 12:46:52.070069 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:46:52.070042    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod287fc015_568a_40cf_bb2a_0f4e46470858.slice/crio-1ce5fff84bdced998f9b7dd2d85533dd34933fe885b6c28ed44bdc50f346ec18 WatchSource:0}: Error finding container 1ce5fff84bdced998f9b7dd2d85533dd34933fe885b6c28ed44bdc50f346ec18: Status 404 returned error can't find the container with id 1ce5fff84bdced998f9b7dd2d85533dd34933fe885b6c28ed44bdc50f346ec18
Sep 15 12:46:52.081669 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:52.081642    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"]
Sep 15 12:46:52.084327 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:46:52.084298    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ec550d_4112_4b55_87b0_5318a1ade216.slice/crio-d221e59b9c57fa4435a4443f4951102b4ae66755533482abb25825878481ed1d WatchSource:0}: Error finding container d221e59b9c57fa4435a4443f4951102b4ae66755533482abb25825878481ed1d: Status 404 returned error can't find the container with id d221e59b9c57fa4435a4443f4951102b4ae66755533482abb25825878481ed1d
Sep 15 12:46:52.479899 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:52.479768    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7" event={"ID":"89ec550d-4112-4b55-87b0-5318a1ade216","Type":"ContainerStarted","Data":"d221e59b9c57fa4435a4443f4951102b4ae66755533482abb25825878481ed1d"}
Sep 15 12:46:52.481455 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:52.481428    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs" event={"ID":"287fc015-568a-40cf-bb2a-0f4e46470858","Type":"ContainerStarted","Data":"89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00"}
Sep 15 12:46:52.481612 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:52.481470    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs" event={"ID":"287fc015-568a-40cf-bb2a-0f4e46470858","Type":"ContainerStarted","Data":"1ce5fff84bdced998f9b7dd2d85533dd34933fe885b6c28ed44bdc50f346ec18"}
Sep 15 12:46:52.481612 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:52.481550    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 12:46:53.488325 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:53.488274    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7" event={"ID":"89ec550d-4112-4b55-87b0-5318a1ade216","Type":"ContainerStarted","Data":"b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2"}
Sep 15 12:46:53.488822 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:53.488577    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:46:53.506122 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:53.506060    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7" podStartSLOduration=2.960566388 podStartE2EDuration="3.506040273s" totalImagesPullingTime="545.473885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:46:53.50300495 +0000 UTC m=+1281.314916393" watchObservedRunningTime="2026-09-15 12:46:53.506040273 +0000 UTC m=+1281.317951716"
Sep 15 12:46:53.506572 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:53.506531    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs" podStartSLOduration=2.506516232 podStartE2EDuration="2.506516232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:46:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:46:52.497631038 +0000 UTC m=+1280.309542481" watchObservedRunningTime="2026-09-15 12:46:53.506516232 +0000 UTC m=+1281.318427678"
Sep 15 12:46:59.500465 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:46:59.500432    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 12:47:03.490863 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:03.490832    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 12:47:17.856191 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.856153    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"]
Sep 15 12:47:17.860236 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.860215    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:17.862898 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.862874    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-17f8f8-openshift-default-dockercfg-2lsm2\""
Sep 15 12:47:17.873984 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.873958    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"]
Sep 15 12:47:17.964458 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.964425    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-data\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:17.964653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.964500    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-certs\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:17.964653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.964534    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-envoy\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:17.964653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.964583    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-credential-socket\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:17.964653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.964606    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-token\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:17.964653 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.964632    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-podinfo\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:17.964899 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.964732    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17735be7-24d6-49ae-8c6a-96ee853b89e2-istiod-ca-cert\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:17.964899 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.964763    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24fmp\" (UniqueName: \"kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-kube-api-access-24fmp\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:17.964899 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:17.964848    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-socket\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066151 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066116    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-socket\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066368 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066184    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-data\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066368 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066246    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-certs\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066368 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066277    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-envoy\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066368 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066325    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-credential-socket\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066368 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066349    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-token\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066380    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-podinfo\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066426    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17735be7-24d6-49ae-8c6a-96ee853b89e2-istiod-ca-cert\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066458    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24fmp\" (UniqueName: \"kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-kube-api-access-24fmp\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066601    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-socket\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066607    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-data\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.066880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.066668    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-credential-socket\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.067045 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.067015    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-certs\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.067216 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.067196    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17735be7-24d6-49ae-8c6a-96ee853b89e2-istiod-ca-cert\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.068599 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.068576    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-envoy\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.068815 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.068796    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-podinfo\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.082731 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.082701    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-token\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.082870 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.082752    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24fmp\" (UniqueName: \"kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-kube-api-access-24fmp\") pod \"e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") " pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.171098 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.171021    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:18.448101 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.448075    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"]
Sep 15 12:47:18.449970 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:47:18.449942    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17735be7_24d6_49ae_8c6a_96ee853b89e2.slice/crio-008b1ae56cda34faa9c3a7ea4b5166e437b3286c6252d55beb555ff266c31e31 WatchSource:0}: Error finding container 008b1ae56cda34faa9c3a7ea4b5166e437b3286c6252d55beb555ff266c31e31: Status 404 returned error can't find the container with id 008b1ae56cda34faa9c3a7ea4b5166e437b3286c6252d55beb555ff266c31e31
Sep 15 12:47:18.452213 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.452180    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:47:18.452318 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.452262    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:47:18.452318 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.452307    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:47:18.630560 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.630523    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" event={"ID":"17735be7-24d6-49ae-8c6a-96ee853b89e2","Type":"ContainerStarted","Data":"fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718"}
Sep 15 12:47:18.630718 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.630593    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" event={"ID":"17735be7-24d6-49ae-8c6a-96ee853b89e2","Type":"ContainerStarted","Data":"008b1ae56cda34faa9c3a7ea4b5166e437b3286c6252d55beb555ff266c31e31"}
Sep 15 12:47:18.653730 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:18.653667    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" podStartSLOduration=1.6536465740000001 podStartE2EDuration="1.653646574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:47:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:47:18.651543383 +0000 UTC m=+1306.463454826" watchObservedRunningTime="2026-09-15 12:47:18.653646574 +0000 UTC m=+1306.465558019"
Sep 15 12:47:19.172143 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:19.172107    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:19.177362 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:19.177337    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:19.637563 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:19.637519    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:19.638618 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:19.638592    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:25.777108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:25.777073    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"]
Sep 15 12:47:25.781034 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:25.781016    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 12:47:25.783732 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:25.783707    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-17f8f8\""
Sep 15 12:47:25.794452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:25.794425    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"]
Sep 15 12:47:25.950757 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:25.950721    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl\" (UID: \"41954fc4-a5d6-45a1-bd0b-4b507d7e598f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 12:47:25.950961 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:25.950857    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6q7x\" (UniqueName: \"kubernetes.io/projected/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-kube-api-access-q6q7x\") pod \"payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl\" (UID: \"41954fc4-a5d6-45a1-bd0b-4b507d7e598f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 12:47:26.051845 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:26.051729    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl\" (UID: \"41954fc4-a5d6-45a1-bd0b-4b507d7e598f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 12:47:26.051845 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:26.051845    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6q7x\" (UniqueName: \"kubernetes.io/projected/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-kube-api-access-q6q7x\") pod \"payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl\" (UID: \"41954fc4-a5d6-45a1-bd0b-4b507d7e598f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 12:47:26.052445 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:26.052423    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl\" (UID: \"41954fc4-a5d6-45a1-bd0b-4b507d7e598f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 12:47:26.059524 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:26.059499    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6q7x\" (UniqueName: \"kubernetes.io/projected/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-kube-api-access-q6q7x\") pod \"payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl\" (UID: \"41954fc4-a5d6-45a1-bd0b-4b507d7e598f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 12:47:26.093213 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:26.093176    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 12:47:26.378238 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:26.378195    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"]
Sep 15 12:47:26.381004 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:47:26.380971    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41954fc4_a5d6_45a1_bd0b_4b507d7e598f.slice/crio-84329b42f90ce800f24ab161575e43370f13f2bc3be7204dad81d8c801e8ff9f WatchSource:0}: Error finding container 84329b42f90ce800f24ab161575e43370f13f2bc3be7204dad81d8c801e8ff9f: Status 404 returned error can't find the container with id 84329b42f90ce800f24ab161575e43370f13f2bc3be7204dad81d8c801e8ff9f
Sep 15 12:47:26.678437 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:26.678338    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl" event={"ID":"41954fc4-a5d6-45a1-bd0b-4b507d7e598f","Type":"ContainerStarted","Data":"e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7"}
Sep 15 12:47:26.678437 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:26.678408    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl" event={"ID":"41954fc4-a5d6-45a1-bd0b-4b507d7e598f","Type":"ContainerStarted","Data":"84329b42f90ce800f24ab161575e43370f13f2bc3be7204dad81d8c801e8ff9f"}
Sep 15 12:47:26.678635 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:26.678455    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 12:47:26.694366 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:26.694316    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl" podStartSLOduration=1.694299115 podStartE2EDuration="1.694299115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:47:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:47:26.692771575 +0000 UTC m=+1314.504683031" watchObservedRunningTime="2026-09-15 12:47:26.694299115 +0000 UTC m=+1314.506210556"
Sep 15 12:47:37.685346 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:37.685316    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 12:47:55.860828 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:55.860792    2586 generic.go:356] "Generic (PLEG): container finished" podID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerID="fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718" exitCode=0
Sep 15 12:47:55.861298 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:55.860858    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" event={"ID":"17735be7-24d6-49ae-8c6a-96ee853b89e2","Type":"ContainerDied","Data":"fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718"}
Sep 15 12:47:55.861490 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:55.861472    2586 scope.go:117] "RemoveContainer" containerID="fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718"
Sep 15 12:47:55.863077 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:55.863041    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:47:55.863156 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:55.863138    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:47:55.863206 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:55.863191    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:47:56.869060 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:56.869019    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" event={"ID":"17735be7-24d6-49ae-8c6a-96ee853b89e2","Type":"ContainerStarted","Data":"426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f"}
Sep 15 12:47:57.171340 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:57.171223    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:47:57.171585 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:57.171558    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 12:47:57.171638 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:57.171615    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 12:47:58.171719 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:58.171676    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 12:47:58.172161 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:58.171739    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 12:47:59.172339 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:59.172301    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 12:47:59.172859 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:47:59.172364    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 12:48:00.171736 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:00.171697    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 12:48:00.171956 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:00.171752    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 12:48:01.172283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:01.172242    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 12:48:01.172689 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:01.172300    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 12:48:02.171739 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:02.171700    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 12:48:02.171957 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:02.171763    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 12:48:03.175805 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:03.175753    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:48:03.176206 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:03.176132    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:48:03.176979 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:03.176959    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 12:48:42.052178 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:42.052141    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"]
Sep 15 12:48:57.071809 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.071750    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n" podUID="29e37ad2-f610-4cd0-920d-196c0fb6a4cc" containerName="main" containerID="cri-o://5e7484aafb0b27a0cf24d330e0eddeddf7e0b36aec38f8eef2b3384aa6b8578f" gracePeriod=45
Sep 15 12:48:57.223390 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.223347    2586 generic.go:356] "Generic (PLEG): container finished" podID="29e37ad2-f610-4cd0-920d-196c0fb6a4cc" containerID="5e7484aafb0b27a0cf24d330e0eddeddf7e0b36aec38f8eef2b3384aa6b8578f" exitCode=0
Sep 15 12:48:57.223574 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.223424    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n" event={"ID":"29e37ad2-f610-4cd0-920d-196c0fb6a4cc","Type":"ContainerDied","Data":"5e7484aafb0b27a0cf24d330e0eddeddf7e0b36aec38f8eef2b3384aa6b8578f"}
Sep 15 12:48:57.338726 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.338701    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:48:57.454492 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.454456    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-dshm\") pod \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") "
Sep 15 12:48:57.454663 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.454547    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8h69\" (UniqueName: \"kubernetes.io/projected/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-kube-api-access-j8h69\") pod \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") "
Sep 15 12:48:57.454663 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.454569    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-model-cache\") pod \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") "
Sep 15 12:48:57.454663 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.454631    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tls-certs\") pod \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") "
Sep 15 12:48:57.454880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.454688    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tmp-dir\") pod \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") "
Sep 15 12:48:57.454880 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.454705    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-home\") pod \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\" (UID: \"29e37ad2-f610-4cd0-920d-196c0fb6a4cc\") "
Sep 15 12:48:57.455006 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.454955    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-model-cache" (OuterVolumeSpecName: "model-cache") pod "29e37ad2-f610-4cd0-920d-196c0fb6a4cc" (UID: "29e37ad2-f610-4cd0-920d-196c0fb6a4cc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:48:57.455065 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.454994    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "29e37ad2-f610-4cd0-920d-196c0fb6a4cc" (UID: "29e37ad2-f610-4cd0-920d-196c0fb6a4cc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:48:57.455065 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.455027    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-home" (OuterVolumeSpecName: "home") pod "29e37ad2-f610-4cd0-920d-196c0fb6a4cc" (UID: "29e37ad2-f610-4cd0-920d-196c0fb6a4cc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:48:57.455169 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.455103    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-model-cache\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:48:57.455169 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.455123    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tmp-dir\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:48:57.456876 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.456848    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-dshm" (OuterVolumeSpecName: "dshm") pod "29e37ad2-f610-4cd0-920d-196c0fb6a4cc" (UID: "29e37ad2-f610-4cd0-920d-196c0fb6a4cc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:48:57.456996 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.456891    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29e37ad2-f610-4cd0-920d-196c0fb6a4cc" (UID: "29e37ad2-f610-4cd0-920d-196c0fb6a4cc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:48:57.456996 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.456907    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-kube-api-access-j8h69" (OuterVolumeSpecName: "kube-api-access-j8h69") pod "29e37ad2-f610-4cd0-920d-196c0fb6a4cc" (UID: "29e37ad2-f610-4cd0-920d-196c0fb6a4cc"). InnerVolumeSpecName "kube-api-access-j8h69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:48:57.556425 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.556383    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8h69\" (UniqueName: \"kubernetes.io/projected/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-kube-api-access-j8h69\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:48:57.556425 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.556418    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-tls-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:48:57.556425 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.556431    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-home\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:48:57.556660 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:57.556442    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29e37ad2-f610-4cd0-920d-196c0fb6a4cc-dshm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:48:58.230292 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:58.230258    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n" event={"ID":"29e37ad2-f610-4cd0-920d-196c0fb6a4cc","Type":"ContainerDied","Data":"3f086044628c67dd5f7210045feb894fec9a4a414b61ac6d1a2ad06f677d2c67"}
Sep 15 12:48:58.230834 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:58.230272    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"
Sep 15 12:48:58.230834 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:58.230310    2586 scope.go:117] "RemoveContainer" containerID="5e7484aafb0b27a0cf24d330e0eddeddf7e0b36aec38f8eef2b3384aa6b8578f"
Sep 15 12:48:58.251651 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:58.251626    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"]
Sep 15 12:48:58.255185 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:58.255162    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-88615e8e/e2e-lifecycle-model-88615e8e-kserve-5c6f465899-nk84n"]
Sep 15 12:48:58.653461 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:48:58.653427    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29e37ad2-f610-4cd0-920d-196c0fb6a4cc" path="/var/lib/kubelet/pods/29e37ad2-f610-4cd0-920d-196c0fb6a4cc/volumes"
Sep 15 12:49:39.397896 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:39.397863    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"]
Sep 15 12:49:39.398834 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:39.398802    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy" containerID="cri-o://e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d" gracePeriod=30
Sep 15 12:49:39.705129 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:39.705045    2586 patch_prober.go:28] interesting pod/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 12:49:39.705129 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:39.705106    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 12:49:44.666557 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.666498    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:49:44.751452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751425    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-data\") pod \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") "
Sep 15 12:49:44.751452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751465    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqtzf\" (UniqueName: \"kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-kube-api-access-mqtzf\") pod \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") "
Sep 15 12:49:44.751644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751505    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-envoy\") pod \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") "
Sep 15 12:49:44.751644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751555    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-credential-socket\") pod \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") "
Sep 15 12:49:44.751644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751577    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-token\") pod \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") "
Sep 15 12:49:44.751644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751600    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-podinfo\") pod \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") "
Sep 15 12:49:44.751644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751619    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-socket\") pod \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") "
Sep 15 12:49:44.751935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751679    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istiod-ca-cert\") pod \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") "
Sep 15 12:49:44.751935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751703    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-data" (OuterVolumeSpecName: "istio-data") pod "2cdaff92-1faf-4d94-9790-8b7c79511ce3" (UID: "2cdaff92-1faf-4d94-9790-8b7c79511ce3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:49:44.751935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751760    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-certs\") pod \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\" (UID: \"2cdaff92-1faf-4d94-9790-8b7c79511ce3\") "
Sep 15 12:49:44.751935 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.751856    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2cdaff92-1faf-4d94-9790-8b7c79511ce3" (UID: "2cdaff92-1faf-4d94-9790-8b7c79511ce3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:49:44.752142 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.752003    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2cdaff92-1faf-4d94-9790-8b7c79511ce3" (UID: "2cdaff92-1faf-4d94-9790-8b7c79511ce3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:49:44.752208 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.752192    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2cdaff92-1faf-4d94-9790-8b7c79511ce3" (UID: "2cdaff92-1faf-4d94-9790-8b7c79511ce3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:49:44.752262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.752215    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2cdaff92-1faf-4d94-9790-8b7c79511ce3" (UID: "2cdaff92-1faf-4d94-9790-8b7c79511ce3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:49:44.752351 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.752326    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:49:44.752491 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.752356    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:49:44.752491 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.752373    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:49:44.753951 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.753929    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2cdaff92-1faf-4d94-9790-8b7c79511ce3" (UID: "2cdaff92-1faf-4d94-9790-8b7c79511ce3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:49:44.753951 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.753927    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2cdaff92-1faf-4d94-9790-8b7c79511ce3" (UID: "2cdaff92-1faf-4d94-9790-8b7c79511ce3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:49:44.753951 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.753947    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-kube-api-access-mqtzf" (OuterVolumeSpecName: "kube-api-access-mqtzf") pod "2cdaff92-1faf-4d94-9790-8b7c79511ce3" (UID: "2cdaff92-1faf-4d94-9790-8b7c79511ce3"). InnerVolumeSpecName "kube-api-access-mqtzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:49:44.754120 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.754002    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-token" (OuterVolumeSpecName: "istio-token") pod "2cdaff92-1faf-4d94-9790-8b7c79511ce3" (UID: "2cdaff92-1faf-4d94-9790-8b7c79511ce3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:49:44.853940 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.853909    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:49:44.853940 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.853935    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:49:44.853940 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.853945    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqtzf\" (UniqueName: \"kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-kube-api-access-mqtzf\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:49:44.854147 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.853954    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:49:44.854147 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.853963    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:49:44.854147 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:44.853970    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cdaff92-1faf-4d94-9790-8b7c79511ce3-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:49:45.491092 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.491056    2586 generic.go:356] "Generic (PLEG): container finished" podID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerID="e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d" exitCode=0
Sep 15 12:49:45.491266 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.491109    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"
Sep 15 12:49:45.491266 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.491140    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" event={"ID":"2cdaff92-1faf-4d94-9790-8b7c79511ce3","Type":"ContainerDied","Data":"e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d"}
Sep 15 12:49:45.491266 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.491201    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx" event={"ID":"2cdaff92-1faf-4d94-9790-8b7c79511ce3","Type":"ContainerDied","Data":"ff5d4b04106b94e2418120c8be24976aa71bfaaaaec93931b8fab3e1bf847d5b"}
Sep 15 12:49:45.491266 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.491225    2586 scope.go:117] "RemoveContainer" containerID="e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d"
Sep 15 12:49:45.501259 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.501238    2586 scope.go:117] "RemoveContainer" containerID="e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c"
Sep 15 12:49:45.509510 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.509490    2586 scope.go:117] "RemoveContainer" containerID="e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d"
Sep 15 12:49:45.509883 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:49:45.509849    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d\": container with ID starting with e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d not found: ID does not exist" containerID="e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d"
Sep 15 12:49:45.510007 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.509888    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d"} err="failed to get container status \"e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d\": rpc error: code = NotFound desc = could not find container \"e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d\": container with ID starting with e98484636dec9f725fd74c5e95e5d639fcada7a7c30c1c1d718bff9c1a8ccb2d not found: ID does not exist"
Sep 15 12:49:45.510007 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.509912    2586 scope.go:117] "RemoveContainer" containerID="e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c"
Sep 15 12:49:45.510252 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:49:45.510230    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c\": container with ID starting with e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c not found: ID does not exist" containerID="e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c"
Sep 15 12:49:45.510367 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.510303    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c"} err="failed to get container status \"e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c\": rpc error: code = NotFound desc = could not find container \"e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c\": container with ID starting with e4983f9c5e1fbbd3f6c3adab59b03e29b6afd0185b3ff0d5791264e45508453c not found: ID does not exist"
Sep 15 12:49:45.513043 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.513022    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"]
Sep 15 12:49:45.516965 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:45.516945    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-88615e8e-openshift-default-696c6cd8b8-bzsrx"]
Sep 15 12:49:46.660154 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:46.660120    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" path="/var/lib/kubelet/pods/2cdaff92-1faf-4d94-9790-8b7c79511ce3/volumes"
Sep 15 12:49:55.442249 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.442208    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"]
Sep 15 12:49:55.442894 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.442875    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy"
Sep 15 12:49:55.442894 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.442895    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy"
Sep 15 12:49:55.442997 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.442906    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29e37ad2-f610-4cd0-920d-196c0fb6a4cc" containerName="main"
Sep 15 12:49:55.442997 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.442912    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="29e37ad2-f610-4cd0-920d-196c0fb6a4cc" containerName="main"
Sep 15 12:49:55.442997 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.442921    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy"
Sep 15 12:49:55.442997 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.442927    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy"
Sep 15 12:49:55.443123 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.443017    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy"
Sep 15 12:49:55.443123 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.443028    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="29e37ad2-f610-4cd0-920d-196c0fb6a4cc" containerName="main"
Sep 15 12:49:55.446302 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.446281    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.449664 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.449644    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-17f8f8\"/\"openshift-service-ca.crt\""
Sep 15 12:49:55.449843 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.449677    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-17f8f8\"/\"default-dockercfg-jhmjz\""
Sep 15 12:49:55.449843 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.449688    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-17f8f8\"/\"auth-test-model-17f8f8-kserve-self-signed-certs\""
Sep 15 12:49:55.449843 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.449763    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-17f8f8\"/\"kube-root-ca.crt\""
Sep 15 12:49:55.458839 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.458816    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"]
Sep 15 12:49:55.567288 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.567251    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tls-certs\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.567288 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.567292    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgn5p\" (UniqueName: \"kubernetes.io/projected/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-kube-api-access-tgn5p\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.567508 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.567360    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tmp-dir\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.567508 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.567414    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-model-cache\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.567508 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.567459    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-dshm\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.567611 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.567508    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-home\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.669014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.668974    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tls-certs\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.669014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.669016    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgn5p\" (UniqueName: \"kubernetes.io/projected/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-kube-api-access-tgn5p\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.669299 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.669155    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tmp-dir\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.669299 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.669207    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-model-cache\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.669299 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.669253    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-dshm\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.669299 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.669292    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-home\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.669610 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.669588    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tmp-dir\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.669610 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.669601    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-model-cache\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.669731 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.669710    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-home\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.671537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.671509    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-dshm\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.671740 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.671721    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tls-certs\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.676823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.676769    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgn5p\" (UniqueName: \"kubernetes.io/projected/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-kube-api-access-tgn5p\") pod \"auth-test-model-17f8f8-kserve-847d748dc9-7dwlm\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:55.759943 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:55.759907    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:49:56.037630 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:56.037600    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"]
Sep 15 12:49:56.039856 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:49:56.039824    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1720f4a8_a28e_4f97_af2c_8013bdbc4f13.slice/crio-948e93e0169517f4dbf3e99ab5f6719629923ba37b61971d280d9e2cd47c4d8f WatchSource:0}: Error finding container 948e93e0169517f4dbf3e99ab5f6719629923ba37b61971d280d9e2cd47c4d8f: Status 404 returned error can't find the container with id 948e93e0169517f4dbf3e99ab5f6719629923ba37b61971d280d9e2cd47c4d8f
Sep 15 12:49:56.558194 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:56.558154    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm" event={"ID":"1720f4a8-a28e-4f97-af2c-8013bdbc4f13","Type":"ContainerStarted","Data":"3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272"}
Sep 15 12:49:56.558606 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:56.558206    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm" event={"ID":"1720f4a8-a28e-4f97-af2c-8013bdbc4f13","Type":"ContainerStarted","Data":"948e93e0169517f4dbf3e99ab5f6719629923ba37b61971d280d9e2cd47c4d8f"}
Sep 15 12:49:56.575254 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:49:56.575202    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm" podStartSLOduration=1.575185321 podStartE2EDuration="1.575185321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:49:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:49:56.574110381 +0000 UTC m=+1464.386021823" watchObservedRunningTime="2026-09-15 12:49:56.575185321 +0000 UTC m=+1464.387096763"
Sep 15 12:50:05.760977 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:05.760935    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:50:05.761571 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:05.761001    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:50:05.774114 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:05.774085    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:50:06.626110 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:06.626078    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:50:09.845454 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.845406    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"]
Sep 15 12:50:09.846258 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.846217    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cdaff92-1faf-4d94-9790-8b7c79511ce3" containerName="istio-proxy"
Sep 15 12:50:09.849936 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.849916    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:09.853228 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.853207    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ee96ec\"/\"distinct-2-w4-ee96ec-kserve-self-signed-certs\""
Sep 15 12:50:09.853383 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.853266    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-ee96ec\"/\"kube-root-ca.crt\""
Sep 15 12:50:09.853383 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.853267    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-ee96ec\"/\"openshift-service-ca.crt\""
Sep 15 12:50:09.853383 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.853268    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ee96ec\"/\"default-dockercfg-gcf26\""
Sep 15 12:50:09.858925 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.858892    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"]
Sep 15 12:50:09.917022 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.916986    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tls-certs\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:09.917210 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.917044    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-home\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:09.917210 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.917129    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tmp-dir\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:09.917210 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.917167    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-model-cache\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:09.917210 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.917201    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6mzc\" (UniqueName: \"kubernetes.io/projected/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-kube-api-access-g6mzc\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:09.917403 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:09.917278    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-dshm\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.018471 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.018424    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-model-cache\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.018674 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.018488    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6mzc\" (UniqueName: \"kubernetes.io/projected/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-kube-api-access-g6mzc\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.018674 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.018533    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-dshm\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.018674 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.018586    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tls-certs\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.018674 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.018650    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-home\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.018925 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.018744    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tmp-dir\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.019174 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.019122    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tmp-dir\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.019309 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.019218    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-home\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.019500 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.019473    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-model-cache\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.021797 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.021732    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-dshm\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.024161 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.024113    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tls-certs\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.027647 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.027524    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6mzc\" (UniqueName: \"kubernetes.io/projected/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-kube-api-access-g6mzc\") pod \"distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") " pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.162938 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.162891    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:10.436970 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.436929    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"]
Sep 15 12:50:10.438080 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:50:10.438051    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcdababb_fa59_4fb3_a4ce_e63e6ad76c9d.slice/crio-126cdee56bddbfd65ecb3f0dbb657aa7df444043305ed9fba55e8d3f67e17ab8 WatchSource:0}: Error finding container 126cdee56bddbfd65ecb3f0dbb657aa7df444043305ed9fba55e8d3f67e17ab8: Status 404 returned error can't find the container with id 126cdee56bddbfd65ecb3f0dbb657aa7df444043305ed9fba55e8d3f67e17ab8
Sep 15 12:50:10.639666 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.639628    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6" event={"ID":"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d","Type":"ContainerStarted","Data":"be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e"}
Sep 15 12:50:10.639666 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.639673    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6" event={"ID":"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d","Type":"ContainerStarted","Data":"126cdee56bddbfd65ecb3f0dbb657aa7df444043305ed9fba55e8d3f67e17ab8"}
Sep 15 12:50:10.657303 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:10.657249    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6" podStartSLOduration=1.657231571 podStartE2EDuration="1.657231571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:50:10.654377101 +0000 UTC m=+1478.466288545" watchObservedRunningTime="2026-09-15 12:50:10.657231571 +0000 UTC m=+1478.469143012"
Sep 15 12:50:20.164035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:20.163970    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:20.164035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:20.164045    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:20.176790 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:20.176748    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:20.711996 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:20.711961    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:50:42.015078 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.015039    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"]
Sep 15 12:50:42.018707 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.018688    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:50:42.021041 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.021022    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-adc0cb-dockercfg-8vvp5\""
Sep 15 12:50:42.021154 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.021106    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-adc0cb\""
Sep 15 12:50:42.029035 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.029011    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"]
Sep 15 12:50:42.063151 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.063117    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzwtd\" (UniqueName: \"kubernetes.io/projected/49204a0d-90a9-419d-9126-2b140892e0fd-kube-api-access-wzwtd\") pod \"payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl\" (UID: \"49204a0d-90a9-419d-9126-2b140892e0fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:50:42.063321 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.063176    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49204a0d-90a9-419d-9126-2b140892e0fd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl\" (UID: \"49204a0d-90a9-419d-9126-2b140892e0fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:50:42.163858 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.163809    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzwtd\" (UniqueName: \"kubernetes.io/projected/49204a0d-90a9-419d-9126-2b140892e0fd-kube-api-access-wzwtd\") pod \"payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl\" (UID: \"49204a0d-90a9-419d-9126-2b140892e0fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:50:42.164082 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.163876    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49204a0d-90a9-419d-9126-2b140892e0fd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl\" (UID: \"49204a0d-90a9-419d-9126-2b140892e0fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:50:42.164541 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.164519    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49204a0d-90a9-419d-9126-2b140892e0fd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl\" (UID: \"49204a0d-90a9-419d-9126-2b140892e0fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:50:42.171523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.171498    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzwtd\" (UniqueName: \"kubernetes.io/projected/49204a0d-90a9-419d-9126-2b140892e0fd-kube-api-access-wzwtd\") pod \"payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl\" (UID: \"49204a0d-90a9-419d-9126-2b140892e0fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:50:42.331573 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.331480    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:50:42.606440 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.606409    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"]
Sep 15 12:50:42.608510 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:50:42.608478    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49204a0d_90a9_419d_9126_2b140892e0fd.slice/crio-34db608fe5863b2c1b359fe2f3955912d8c1522616f9f8ba360e4b8800cd4f1a WatchSource:0}: Error finding container 34db608fe5863b2c1b359fe2f3955912d8c1522616f9f8ba360e4b8800cd4f1a: Status 404 returned error can't find the container with id 34db608fe5863b2c1b359fe2f3955912d8c1522616f9f8ba360e4b8800cd4f1a
Sep 15 12:50:42.832617 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.832580    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl" event={"ID":"49204a0d-90a9-419d-9126-2b140892e0fd","Type":"ContainerStarted","Data":"75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9"}
Sep 15 12:50:42.832840 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.832641    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl" event={"ID":"49204a0d-90a9-419d-9126-2b140892e0fd","Type":"ContainerStarted","Data":"34db608fe5863b2c1b359fe2f3955912d8c1522616f9f8ba360e4b8800cd4f1a"}
Sep 15 12:50:42.832840 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.832749    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:50:42.849338 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:42.849287    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl" podStartSLOduration=1.8492702269999999 podStartE2EDuration="1.849270227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:50:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:50:42.847926538 +0000 UTC m=+1510.659838007" watchObservedRunningTime="2026-09-15 12:50:42.849270227 +0000 UTC m=+1510.661181669"
Sep 15 12:50:53.840463 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:50:53.840429    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:51:07.255396 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.255139    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"]
Sep 15 12:51:07.258535 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.258516    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.260938 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.260914    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-25e7f8-openshift-default-dockercfg-tvjxx\""
Sep 15 12:51:07.269624 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.269599    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"]
Sep 15 12:51:07.449499 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.449459    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-istio-token\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.449689 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.449520    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-credential-socket\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.449689 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.449563    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-socket\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.449689 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.449619    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-data\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.449689 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.449652    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whjgm\" (UniqueName: \"kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-kube-api-access-whjgm\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.449907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.449700    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/863ff6d0-07ed-445a-8744-db60637b94eb-istiod-ca-cert\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.449907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.449726    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/863ff6d0-07ed-445a-8744-db60637b94eb-istio-podinfo\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.449907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.449790    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-envoy\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.449907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.449820    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-certs\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.551357 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.551263    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-istio-token\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.551357 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.551327    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-credential-socket\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.551595 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.551381    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-socket\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.551595 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.551525    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-data\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.551595 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.551582    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whjgm\" (UniqueName: \"kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-kube-api-access-whjgm\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.551726 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.551642    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/863ff6d0-07ed-445a-8744-db60637b94eb-istiod-ca-cert\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.551726 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.551685    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/863ff6d0-07ed-445a-8744-db60637b94eb-istio-podinfo\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.551850 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.551730    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-envoy\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.551850 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.551764    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-certs\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.551850 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.551822    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-credential-socket\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.552048 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.552027    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-socket\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.552114 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.552076    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-data\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.552251 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.552229    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-certs\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.552709 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.552679    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/863ff6d0-07ed-445a-8744-db60637b94eb-istiod-ca-cert\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.554281 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.554252    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-envoy\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.554402 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.554383    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/863ff6d0-07ed-445a-8744-db60637b94eb-istio-podinfo\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.558563 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.558533    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-istio-token\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.558804 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.558751    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whjgm\" (UniqueName: \"kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-kube-api-access-whjgm\") pod \"e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") " pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.569683 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.569659    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:07.852052 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.852025    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"]
Sep 15 12:51:07.854303 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:51:07.854272    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863ff6d0_07ed_445a_8744_db60637b94eb.slice/crio-240e31dcd1e9b5e633cf5cf39e9d8e82ca1580859cb6d119048f332ea51e1653 WatchSource:0}: Error finding container 240e31dcd1e9b5e633cf5cf39e9d8e82ca1580859cb6d119048f332ea51e1653: Status 404 returned error can't find the container with id 240e31dcd1e9b5e633cf5cf39e9d8e82ca1580859cb6d119048f332ea51e1653
Sep 15 12:51:07.856872 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.856828    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:51:07.857007 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.856899    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:51:07.857007 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.856930    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:51:07.981541 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.981496    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" event={"ID":"863ff6d0-07ed-445a-8744-db60637b94eb","Type":"ContainerStarted","Data":"2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91"}
Sep 15 12:51:07.981835 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:07.981814    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" event={"ID":"863ff6d0-07ed-445a-8744-db60637b94eb","Type":"ContainerStarted","Data":"240e31dcd1e9b5e633cf5cf39e9d8e82ca1580859cb6d119048f332ea51e1653"}
Sep 15 12:51:08.018121 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:08.018049    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" podStartSLOduration=1.018025314 podStartE2EDuration="1.018025314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:51:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:51:08.014847131 +0000 UTC m=+1535.826758574" watchObservedRunningTime="2026-09-15 12:51:08.018025314 +0000 UTC m=+1535.829936758"
Sep 15 12:51:08.570485 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:08.570447    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:08.575959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:08.575930    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:08.987716 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:08.987684    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:08.988604 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:08.988585    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:40.352539 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.352498    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"]
Sep 15 12:51:40.357220 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.357194    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.361297 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.361271    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-11f865\"/\"default-dockercfg-9fd5c\""
Sep 15 12:51:40.362017 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.361523    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-11f865\"/\"openshift-service-ca.crt\""
Sep 15 12:51:40.362017 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.361747    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-11f865\"/\"embedding-w0-11f865-kserve-self-signed-certs\""
Sep 15 12:51:40.362017 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.361914    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-11f865\"/\"kube-root-ca.crt\""
Sep 15 12:51:40.364195 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.364135    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"]
Sep 15 12:51:40.409334 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.409298    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-tmp-dir\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.409518 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.409353    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81eb1888-e600-4fc5-95d3-798b6d21675d-tls-certs\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.409518 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.409458    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzwf8\" (UniqueName: \"kubernetes.io/projected/81eb1888-e600-4fc5-95d3-798b6d21675d-kube-api-access-rzwf8\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.409621 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.409525    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-dshm\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.409668 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.409618    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-model-cache\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.409668 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.409657    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-home\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.511481 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.511439    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzwf8\" (UniqueName: \"kubernetes.io/projected/81eb1888-e600-4fc5-95d3-798b6d21675d-kube-api-access-rzwf8\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.511665 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.511539    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-dshm\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.511665 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.511601    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-model-cache\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.511665 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.511642    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-home\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.511928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.511687    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-tmp-dir\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.511928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.511736    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81eb1888-e600-4fc5-95d3-798b6d21675d-tls-certs\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.512236 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.512149    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-model-cache\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.512236 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.512178    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-home\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.512236 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.512212    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-tmp-dir\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.514540 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.514517    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-dshm\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.515277 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.515249    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81eb1888-e600-4fc5-95d3-798b6d21675d-tls-certs\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.520581 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.520523    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzwf8\" (UniqueName: \"kubernetes.io/projected/81eb1888-e600-4fc5-95d3-798b6d21675d-kube-api-access-rzwf8\") pod \"embedding-w0-11f865-kserve-5dd66f6879-trkjh\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") " pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.671942 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.671861    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:40.984467 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:40.984439    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"]
Sep 15 12:51:40.986070 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:51:40.986038    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81eb1888_e600_4fc5_95d3_798b6d21675d.slice/crio-7be7793f49d2871880f570ad91dda400477b7a926867569ac0c4b3f01a81d7b5 WatchSource:0}: Error finding container 7be7793f49d2871880f570ad91dda400477b7a926867569ac0c4b3f01a81d7b5: Status 404 returned error can't find the container with id 7be7793f49d2871880f570ad91dda400477b7a926867569ac0c4b3f01a81d7b5
Sep 15 12:51:41.181944 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:41.181909    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh" event={"ID":"81eb1888-e600-4fc5-95d3-798b6d21675d","Type":"ContainerStarted","Data":"8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c"}
Sep 15 12:51:41.182127 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:41.181953    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh" event={"ID":"81eb1888-e600-4fc5-95d3-798b6d21675d","Type":"ContainerStarted","Data":"7be7793f49d2871880f570ad91dda400477b7a926867569ac0c4b3f01a81d7b5"}
Sep 15 12:51:41.204697 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:41.204636    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh" podStartSLOduration=1.204620982 podStartE2EDuration="1.204620982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:51:41.202899117 +0000 UTC m=+1569.014810561" watchObservedRunningTime="2026-09-15 12:51:41.204620982 +0000 UTC m=+1569.016532424"
Sep 15 12:51:50.672888 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:50.672852    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:50.673394 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:50.672920    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:50.685910 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:50.685881    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:51.263026 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:51.262993    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:51:53.264837 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:53.264797    2586 generic.go:356] "Generic (PLEG): container finished" podID="863ff6d0-07ed-445a-8744-db60637b94eb" containerID="2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91" exitCode=0
Sep 15 12:51:53.264837 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:53.264808    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" event={"ID":"863ff6d0-07ed-445a-8744-db60637b94eb","Type":"ContainerDied","Data":"2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91"}
Sep 15 12:51:53.265338 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:53.265286    2586 scope.go:117] "RemoveContainer" containerID="2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91"
Sep 15 12:51:53.266521 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:53.266490    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:51:53.266578 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:53.266560    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:51:53.266626 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:53.266593    2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:51:53.988057 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:53.988020    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:54.273632 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:54.273532    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" event={"ID":"863ff6d0-07ed-445a-8744-db60637b94eb","Type":"ContainerStarted","Data":"ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816"}
Sep 15 12:51:54.570549 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:54.570454    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:54.571854 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:54.571830    2586 patch_prober.go:28] interesting pod/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 15 12:51:54.571968 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:54.571922    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 15 12:51:55.570605 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:55.570565    2586 patch_prober.go:28] interesting pod/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 15 12:51:55.571037 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:55.570624    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 15 12:51:56.570142 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:56.570100    2586 patch_prober.go:28] interesting pod/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 15 12:51:56.570352 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:56.570168    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 15 12:51:57.570705 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:57.570670    2586 patch_prober.go:28] interesting pod/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 15 12:51:57.571120 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:57.570727    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 15 12:51:58.570964 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:58.570928    2586 patch_prober.go:28] interesting pod/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 15 12:51:58.571364 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:58.570984    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 15 12:51:58.708378 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:58.708346    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"]
Sep 15 12:51:58.708895 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:58.708861    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl" podUID="49204a0d-90a9-419d-9126-2b140892e0fd" containerName="payload-processing" containerID="cri-o://75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9" gracePeriod=30
Sep 15 12:51:59.064290 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.064261    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:51:59.239697 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.239664    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzwtd\" (UniqueName: \"kubernetes.io/projected/49204a0d-90a9-419d-9126-2b140892e0fd-kube-api-access-wzwtd\") pod \"49204a0d-90a9-419d-9126-2b140892e0fd\" (UID: \"49204a0d-90a9-419d-9126-2b140892e0fd\") "
Sep 15 12:51:59.239697 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.239701    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49204a0d-90a9-419d-9126-2b140892e0fd-plugins-config-volume\") pod \"49204a0d-90a9-419d-9126-2b140892e0fd\" (UID: \"49204a0d-90a9-419d-9126-2b140892e0fd\") "
Sep 15 12:51:59.240101 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.240076    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49204a0d-90a9-419d-9126-2b140892e0fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49204a0d-90a9-419d-9126-2b140892e0fd" (UID: "49204a0d-90a9-419d-9126-2b140892e0fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:51:59.241886 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.241859    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49204a0d-90a9-419d-9126-2b140892e0fd-kube-api-access-wzwtd" (OuterVolumeSpecName: "kube-api-access-wzwtd") pod "49204a0d-90a9-419d-9126-2b140892e0fd" (UID: "49204a0d-90a9-419d-9126-2b140892e0fd"). InnerVolumeSpecName "kube-api-access-wzwtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:51:59.305749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.305710    2586 generic.go:356] "Generic (PLEG): container finished" podID="49204a0d-90a9-419d-9126-2b140892e0fd" containerID="75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9" exitCode=0
Sep 15 12:51:59.305749 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.305758    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"
Sep 15 12:51:59.306014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.305835    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl" event={"ID":"49204a0d-90a9-419d-9126-2b140892e0fd","Type":"ContainerDied","Data":"75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9"}
Sep 15 12:51:59.306014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.305901    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl" event={"ID":"49204a0d-90a9-419d-9126-2b140892e0fd","Type":"ContainerDied","Data":"34db608fe5863b2c1b359fe2f3955912d8c1522616f9f8ba360e4b8800cd4f1a"}
Sep 15 12:51:59.306014 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.305936    2586 scope.go:117] "RemoveContainer" containerID="75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9"
Sep 15 12:51:59.317259 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.317225    2586 scope.go:117] "RemoveContainer" containerID="75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9"
Sep 15 12:51:59.317684 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:51:59.317641    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9\": container with ID starting with 75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9 not found: ID does not exist" containerID="75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9"
Sep 15 12:51:59.317684 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.317675    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9"} err="failed to get container status \"75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9\": rpc error: code = NotFound desc = could not find container \"75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9\": container with ID starting with 75f3ff20d522ae561d3bf4a65ad36d27f9b0aa59b72381053af001fcfe0179c9 not found: ID does not exist"
Sep 15 12:51:59.331613 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.331581    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"]
Sep 15 12:51:59.335505 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.335476    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-adc0cb-b5d8b885c-glhbl"]
Sep 15 12:51:59.341700 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.341669    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzwtd\" (UniqueName: \"kubernetes.io/projected/49204a0d-90a9-419d-9126-2b140892e0fd-kube-api-access-wzwtd\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:51:59.341813 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.341706    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49204a0d-90a9-419d-9126-2b140892e0fd-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:51:59.574588 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.574514    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:59.574959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.574870    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:51:59.575540 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:51:59.575521    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:52:00.659204 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:00.659170    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49204a0d-90a9-419d-9126-2b140892e0fd" path="/var/lib/kubelet/pods/49204a0d-90a9-419d-9126-2b140892e0fd/volumes"
Sep 15 12:52:28.108865 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:28.108760    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"]
Sep 15 12:52:28.109434 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:28.109059    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy" containerID="cri-o://ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816" gracePeriod=30
Sep 15 12:52:29.578020 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:29.577983    2586 patch_prober.go:28] interesting pod/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 15 12:52:29.578416 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:29.578051    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 12:52:33.365404 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.365377    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:52:33.436643 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.436615    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-credential-socket\") pod \"863ff6d0-07ed-445a-8744-db60637b94eb\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") "
Sep 15 12:52:33.436643 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.436649    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/863ff6d0-07ed-445a-8744-db60637b94eb-istio-podinfo\") pod \"863ff6d0-07ed-445a-8744-db60637b94eb\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") "
Sep 15 12:52:33.436907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.436713    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/863ff6d0-07ed-445a-8744-db60637b94eb-istiod-ca-cert\") pod \"863ff6d0-07ed-445a-8744-db60637b94eb\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") "
Sep 15 12:52:33.436907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.436737    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-istio-token\") pod \"863ff6d0-07ed-445a-8744-db60637b94eb\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") "
Sep 15 12:52:33.436907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.436767    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-envoy\") pod \"863ff6d0-07ed-445a-8744-db60637b94eb\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") "
Sep 15 12:52:33.436907 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.436862    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-data\") pod \"863ff6d0-07ed-445a-8744-db60637b94eb\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") "
Sep 15 12:52:33.437132 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.436897    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "863ff6d0-07ed-445a-8744-db60637b94eb" (UID: "863ff6d0-07ed-445a-8744-db60637b94eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:52:33.437132 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.436930    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-certs\") pod \"863ff6d0-07ed-445a-8744-db60637b94eb\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") "
Sep 15 12:52:33.437132 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.436970    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whjgm\" (UniqueName: \"kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-kube-api-access-whjgm\") pod \"863ff6d0-07ed-445a-8744-db60637b94eb\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") "
Sep 15 12:52:33.437132 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.437020    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-socket\") pod \"863ff6d0-07ed-445a-8744-db60637b94eb\" (UID: \"863ff6d0-07ed-445a-8744-db60637b94eb\") "
Sep 15 12:52:33.437329 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.437185    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-data" (OuterVolumeSpecName: "istio-data") pod "863ff6d0-07ed-445a-8744-db60637b94eb" (UID: "863ff6d0-07ed-445a-8744-db60637b94eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:52:33.437329 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.437200    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "863ff6d0-07ed-445a-8744-db60637b94eb" (UID: "863ff6d0-07ed-445a-8744-db60637b94eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:52:33.437329 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.437209    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/863ff6d0-07ed-445a-8744-db60637b94eb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "863ff6d0-07ed-445a-8744-db60637b94eb" (UID: "863ff6d0-07ed-445a-8744-db60637b94eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:52:33.437465 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.437354    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "863ff6d0-07ed-445a-8744-db60637b94eb" (UID: "863ff6d0-07ed-445a-8744-db60637b94eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:52:33.437570 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.437550    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/863ff6d0-07ed-445a-8744-db60637b94eb-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:33.437613 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.437576    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:33.437613 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.437590    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:33.437613 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.437603    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:33.437760 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.437616    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:33.439233 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.439204    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-istio-token" (OuterVolumeSpecName: "istio-token") pod "863ff6d0-07ed-445a-8744-db60637b94eb" (UID: "863ff6d0-07ed-445a-8744-db60637b94eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:52:33.439409 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.439245    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-kube-api-access-whjgm" (OuterVolumeSpecName: "kube-api-access-whjgm") pod "863ff6d0-07ed-445a-8744-db60637b94eb" (UID: "863ff6d0-07ed-445a-8744-db60637b94eb"). InnerVolumeSpecName "kube-api-access-whjgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:52:33.439473 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.439441    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/863ff6d0-07ed-445a-8744-db60637b94eb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "863ff6d0-07ed-445a-8744-db60637b94eb" (UID: "863ff6d0-07ed-445a-8744-db60637b94eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:52:33.439590 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.439567    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "863ff6d0-07ed-445a-8744-db60637b94eb" (UID: "863ff6d0-07ed-445a-8744-db60637b94eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:52:33.508088 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.508056    2586 generic.go:356] "Generic (PLEG): container finished" podID="863ff6d0-07ed-445a-8744-db60637b94eb" containerID="ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816" exitCode=0
Sep 15 12:52:33.508231 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.508101    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"
Sep 15 12:52:33.508231 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.508142    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" event={"ID":"863ff6d0-07ed-445a-8744-db60637b94eb","Type":"ContainerDied","Data":"ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816"}
Sep 15 12:52:33.508231 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.508226    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc" event={"ID":"863ff6d0-07ed-445a-8744-db60637b94eb","Type":"ContainerDied","Data":"240e31dcd1e9b5e633cf5cf39e9d8e82ca1580859cb6d119048f332ea51e1653"}
Sep 15 12:52:33.508340 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.508252    2586 scope.go:117] "RemoveContainer" containerID="ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816"
Sep 15 12:52:33.521275 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.521256    2586 scope.go:117] "RemoveContainer" containerID="2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91"
Sep 15 12:52:33.530923 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.530882    2586 scope.go:117] "RemoveContainer" containerID="ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816"
Sep 15 12:52:33.531212 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:52:33.531187    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816\": container with ID starting with ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816 not found: ID does not exist" containerID="ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816"
Sep 15 12:52:33.531306 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.531219    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816"} err="failed to get container status \"ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816\": rpc error: code = NotFound desc = could not find container \"ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816\": container with ID starting with ae25be0324d568cf6abae0f5c7d5d18e33fad681b5102b1cdef81222d3645816 not found: ID does not exist"
Sep 15 12:52:33.531306 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.531237    2586 scope.go:117] "RemoveContainer" containerID="2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91"
Sep 15 12:52:33.531531 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:52:33.531513    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91\": container with ID starting with 2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91 not found: ID does not exist" containerID="2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91"
Sep 15 12:52:33.531587 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.531544    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91"} err="failed to get container status \"2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91\": rpc error: code = NotFound desc = could not find container \"2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91\": container with ID starting with 2687cd03f46da7f0c25a2b9a52c004f7839548273d53bbaae51bdf3beaf11a91 not found: ID does not exist"
Sep 15 12:52:33.532863 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.532845    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"]
Sep 15 12:52:33.537788 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.537760    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-25e7f8-openshift-default-54c956567b-wx8kc"]
Sep 15 12:52:33.538293 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.538275    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whjgm\" (UniqueName: \"kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-kube-api-access-whjgm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:33.538336 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.538297    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/863ff6d0-07ed-445a-8744-db60637b94eb-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:33.538336 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.538307    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/863ff6d0-07ed-445a-8744-db60637b94eb-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:33.538336 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:33.538316    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/863ff6d0-07ed-445a-8744-db60637b94eb-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:34.653862 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:34.653825    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" path="/var/lib/kubelet/pods/863ff6d0-07ed-445a-8744-db60637b94eb/volumes"
Sep 15 12:52:39.186169 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.186130    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"]
Sep 15 12:52:39.186692 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.186472    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf" podUID="fafcfc94-94c1-45ee-a507-c26613679560" containerName="payload-processing" containerID="cri-o://e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1" gracePeriod=30
Sep 15 12:52:39.195919 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.195892    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"]
Sep 15 12:52:39.196333 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.196307    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22" podUID="a1f2c109-17c9-4045-8e75-aaf57109caf1" containerName="payload-pre-processing" containerID="cri-o://7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191" gracePeriod=30
Sep 15 12:52:39.473219 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.473192    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:52:39.537881 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.537490    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:52:39.548890 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.548855    2586 generic.go:356] "Generic (PLEG): container finished" podID="a1f2c109-17c9-4045-8e75-aaf57109caf1" containerID="7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191" exitCode=0
Sep 15 12:52:39.549097 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.548903    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"
Sep 15 12:52:39.549097 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.548938    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22" event={"ID":"a1f2c109-17c9-4045-8e75-aaf57109caf1","Type":"ContainerDied","Data":"7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191"}
Sep 15 12:52:39.549097 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.549004    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22" event={"ID":"a1f2c109-17c9-4045-8e75-aaf57109caf1","Type":"ContainerDied","Data":"fe6e11b9f3d0c7c2bf907eb0f3460da80e85db18bc647c20e1592c4a60f6b3de"}
Sep 15 12:52:39.549097 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.549044    2586 scope.go:117] "RemoveContainer" containerID="7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191"
Sep 15 12:52:39.550478 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.550458    2586 generic.go:356] "Generic (PLEG): container finished" podID="fafcfc94-94c1-45ee-a507-c26613679560" containerID="e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1" exitCode=0
Sep 15 12:52:39.550576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.550501    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"
Sep 15 12:52:39.550576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.550544    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf" event={"ID":"fafcfc94-94c1-45ee-a507-c26613679560","Type":"ContainerDied","Data":"e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1"}
Sep 15 12:52:39.550688 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.550594    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf" event={"ID":"fafcfc94-94c1-45ee-a507-c26613679560","Type":"ContainerDied","Data":"09041ba69396b848b8f3c121c0ed3828416b759f19a0d506116ce6d11cbb49e2"}
Sep 15 12:52:39.559113 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.559090    2586 scope.go:117] "RemoveContainer" containerID="7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191"
Sep 15 12:52:39.559434 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:52:39.559409    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191\": container with ID starting with 7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191 not found: ID does not exist" containerID="7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191"
Sep 15 12:52:39.559525 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.559439    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191"} err="failed to get container status \"7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191\": rpc error: code = NotFound desc = could not find container \"7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191\": container with ID starting with 7914848e4298825e874d1509ef2baa1bcc41809514b91812b21d793fa9132191 not found: ID does not exist"
Sep 15 12:52:39.559525 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.559456    2586 scope.go:117] "RemoveContainer" containerID="e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1"
Sep 15 12:52:39.570439 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.570409    2586 scope.go:117] "RemoveContainer" containerID="e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1"
Sep 15 12:52:39.570792 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:52:39.570760    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1\": container with ID starting with e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1 not found: ID does not exist" containerID="e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1"
Sep 15 12:52:39.570869 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.570802    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1"} err="failed to get container status \"e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1\": rpc error: code = NotFound desc = could not find container \"e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1\": container with ID starting with e7539fda8b6cdef7e25a9c651993dfb2ab2845678f008a2ca2f0b7fad2dd97d1 not found: ID does not exist"
Sep 15 12:52:39.603193 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.603156    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1f2c109-17c9-4045-8e75-aaf57109caf1-plugins-config-volume\") pod \"a1f2c109-17c9-4045-8e75-aaf57109caf1\" (UID: \"a1f2c109-17c9-4045-8e75-aaf57109caf1\") "
Sep 15 12:52:39.603387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.603255    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fafcfc94-94c1-45ee-a507-c26613679560-plugins-config-volume\") pod \"fafcfc94-94c1-45ee-a507-c26613679560\" (UID: \"fafcfc94-94c1-45ee-a507-c26613679560\") "
Sep 15 12:52:39.603387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.603286    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p84b7\" (UniqueName: \"kubernetes.io/projected/fafcfc94-94c1-45ee-a507-c26613679560-kube-api-access-p84b7\") pod \"fafcfc94-94c1-45ee-a507-c26613679560\" (UID: \"fafcfc94-94c1-45ee-a507-c26613679560\") "
Sep 15 12:52:39.603387 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.603374    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjclp\" (UniqueName: \"kubernetes.io/projected/a1f2c109-17c9-4045-8e75-aaf57109caf1-kube-api-access-bjclp\") pod \"a1f2c109-17c9-4045-8e75-aaf57109caf1\" (UID: \"a1f2c109-17c9-4045-8e75-aaf57109caf1\") "
Sep 15 12:52:39.603644 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.603614    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1f2c109-17c9-4045-8e75-aaf57109caf1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1f2c109-17c9-4045-8e75-aaf57109caf1" (UID: "a1f2c109-17c9-4045-8e75-aaf57109caf1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:52:39.603949 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.603888    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1f2c109-17c9-4045-8e75-aaf57109caf1-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:39.604093 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.604008    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fafcfc94-94c1-45ee-a507-c26613679560-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fafcfc94-94c1-45ee-a507-c26613679560" (UID: "fafcfc94-94c1-45ee-a507-c26613679560"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:52:39.606131 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.606101    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1f2c109-17c9-4045-8e75-aaf57109caf1-kube-api-access-bjclp" (OuterVolumeSpecName: "kube-api-access-bjclp") pod "a1f2c109-17c9-4045-8e75-aaf57109caf1" (UID: "a1f2c109-17c9-4045-8e75-aaf57109caf1"). InnerVolumeSpecName "kube-api-access-bjclp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:52:39.606238 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.606215    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fafcfc94-94c1-45ee-a507-c26613679560-kube-api-access-p84b7" (OuterVolumeSpecName: "kube-api-access-p84b7") pod "fafcfc94-94c1-45ee-a507-c26613679560" (UID: "fafcfc94-94c1-45ee-a507-c26613679560"). InnerVolumeSpecName "kube-api-access-p84b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:52:39.704991 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.704887    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fafcfc94-94c1-45ee-a507-c26613679560-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:39.704991 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.704920    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p84b7\" (UniqueName: \"kubernetes.io/projected/fafcfc94-94c1-45ee-a507-c26613679560-kube-api-access-p84b7\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:39.704991 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.704934    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjclp\" (UniqueName: \"kubernetes.io/projected/a1f2c109-17c9-4045-8e75-aaf57109caf1-kube-api-access-bjclp\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:39.873414 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.873382    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"]
Sep 15 12:52:39.876713 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.876688    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1ba7b-6c8f7dc88f-j2nwf"]
Sep 15 12:52:39.885853 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.885827    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"]
Sep 15 12:52:39.889642 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:39.889617    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1ba7b-5ff75f4fd6-hfq22"]
Sep 15 12:52:40.656336 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:40.656153    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1f2c109-17c9-4045-8e75-aaf57109caf1" path="/var/lib/kubelet/pods/a1f2c109-17c9-4045-8e75-aaf57109caf1/volumes"
Sep 15 12:52:40.656900 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:40.656571    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fafcfc94-94c1-45ee-a507-c26613679560" path="/var/lib/kubelet/pods/fafcfc94-94c1-45ee-a507-c26613679560/volumes"
Sep 15 12:52:56.774590 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:56.774550    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"]
Sep 15 12:52:56.775139 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:56.774887    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc" podUID="eaa5c813-a6be-42dd-a3d6-2f33c78699f0" containerName="maas-api" containerID="cri-o://77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b" gracePeriod=30
Sep 15 12:52:57.048344 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.048320    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:52:57.185352 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.185323    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-metrics-certs\") pod \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") "
Sep 15 12:52:57.185537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.185369    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-maas-api-tls\") pod \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") "
Sep 15 12:52:57.185537 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.185390    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-openshift-service-ca\") pod \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") "
Sep 15 12:52:57.185627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.185614    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fr2d\" (UniqueName: \"kubernetes.io/projected/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-kube-api-access-8fr2d\") pod \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\" (UID: \"eaa5c813-a6be-42dd-a3d6-2f33c78699f0\") "
Sep 15 12:52:57.185791 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.185745    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eaa5c813-a6be-42dd-a3d6-2f33c78699f0" (UID: "eaa5c813-a6be-42dd-a3d6-2f33c78699f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:52:57.186023 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.186005    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-openshift-service-ca\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:57.187998 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.187973    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eaa5c813-a6be-42dd-a3d6-2f33c78699f0" (UID: "eaa5c813-a6be-42dd-a3d6-2f33c78699f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:52:57.188115 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.188023    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eaa5c813-a6be-42dd-a3d6-2f33c78699f0" (UID: "eaa5c813-a6be-42dd-a3d6-2f33c78699f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:52:57.188189 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.188103    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-kube-api-access-8fr2d" (OuterVolumeSpecName: "kube-api-access-8fr2d") pod "eaa5c813-a6be-42dd-a3d6-2f33c78699f0" (UID: "eaa5c813-a6be-42dd-a3d6-2f33c78699f0"). InnerVolumeSpecName "kube-api-access-8fr2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:52:57.287108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.287019    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fr2d\" (UniqueName: \"kubernetes.io/projected/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-kube-api-access-8fr2d\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:57.287108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.287050    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:57.287108 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.287060    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5c813-a6be-42dd-a3d6-2f33c78699f0-maas-api-tls\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:52:57.657176 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.657078    2586 generic.go:356] "Generic (PLEG): container finished" podID="eaa5c813-a6be-42dd-a3d6-2f33c78699f0" containerID="77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b" exitCode=0
Sep 15 12:52:57.657176 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.657131    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"
Sep 15 12:52:57.657176 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.657161    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc" event={"ID":"eaa5c813-a6be-42dd-a3d6-2f33c78699f0","Type":"ContainerDied","Data":"77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b"}
Sep 15 12:52:57.657446 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.657224    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc" event={"ID":"eaa5c813-a6be-42dd-a3d6-2f33c78699f0","Type":"ContainerDied","Data":"a759e299c9bcb98bcfa8e58c6158e72b07b49395dae75de2df7af0ab3039a549"}
Sep 15 12:52:57.657446 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.657248    2586 scope.go:117] "RemoveContainer" containerID="77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b"
Sep 15 12:52:57.667433 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.667416    2586 scope.go:117] "RemoveContainer" containerID="77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b"
Sep 15 12:52:57.667710 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:52:57.667693    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b\": container with ID starting with 77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b not found: ID does not exist" containerID="77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b"
Sep 15 12:52:57.667799 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.667721    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b"} err="failed to get container status \"77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b\": rpc error: code = NotFound desc = could not find container \"77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b\": container with ID starting with 77d4af1b4ed336c76638f129258c6f7a18a5e98ccceb279dc78e000849055d5b not found: ID does not exist"
Sep 15 12:52:57.679452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.679423    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"]
Sep 15 12:52:57.681649 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:57.681623    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9e7a1-f986d77dd-8wlcc"]
Sep 15 12:52:58.653360 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:52:58.653328    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaa5c813-a6be-42dd-a3d6-2f33c78699f0" path="/var/lib/kubelet/pods/eaa5c813-a6be-42dd-a3d6-2f33c78699f0/volumes"
Sep 15 12:53:03.440314 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:03.440277    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"]
Sep 15 12:53:03.441305 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:03.440560    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy" containerID="cri-o://eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117" gracePeriod=30
Sep 15 12:53:07.958146 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:07.958106    2586 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 12:53:07.958529 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:07.958178    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 12:53:08.717219 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.717152    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:53:08.722417 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.722391    2586 generic.go:356] "Generic (PLEG): container finished" podID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerID="eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117" exitCode=0
Sep 15 12:53:08.722548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.722435    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"
Sep 15 12:53:08.722548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.722464    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" event={"ID":"3a424e6d-cf84-4965-8142-1998d0fd624a","Type":"ContainerDied","Data":"eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117"}
Sep 15 12:53:08.722548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.722517    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd" event={"ID":"3a424e6d-cf84-4965-8142-1998d0fd624a","Type":"ContainerDied","Data":"abc45e976edf43c1673c63f25718538375c028fa387cc22f0d5e444a69bfae6a"}
Sep 15 12:53:08.722548 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.722541    2586 scope.go:117] "RemoveContainer" containerID="eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117"
Sep 15 12:53:08.731636 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.731614    2586 scope.go:117] "RemoveContainer" containerID="a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38"
Sep 15 12:53:08.742636 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.742603    2586 scope.go:117] "RemoveContainer" containerID="eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117"
Sep 15 12:53:08.742981 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:53:08.742957    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117\": container with ID starting with eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117 not found: ID does not exist" containerID="eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117"
Sep 15 12:53:08.743097 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.742993    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117"} err="failed to get container status \"eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117\": rpc error: code = NotFound desc = could not find container \"eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117\": container with ID starting with eacb71c2d0cdff6402c8b2cadee0a1b9d9e4f7783eb2027c05d6e84ea1627117 not found: ID does not exist"
Sep 15 12:53:08.743097 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.743016    2586 scope.go:117] "RemoveContainer" containerID="a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38"
Sep 15 12:53:08.743329 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:53:08.743313    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38\": container with ID starting with a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38 not found: ID does not exist" containerID="a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38"
Sep 15 12:53:08.743382 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.743338    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38"} err="failed to get container status \"a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38\": rpc error: code = NotFound desc = could not find container \"a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38\": container with ID starting with a6b671b19cef528514efda19e8444af6cd0302eb99a14df8e89d52d8acd0fb38 not found: ID does not exist"
Sep 15 12:53:08.806940 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.806907    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-podinfo\") pod \"3a424e6d-cf84-4965-8142-1998d0fd624a\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") "
Sep 15 12:53:08.807133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.806968    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jgfq\" (UniqueName: \"kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-kube-api-access-9jgfq\") pod \"3a424e6d-cf84-4965-8142-1998d0fd624a\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") "
Sep 15 12:53:08.807133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807004    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-data\") pod \"3a424e6d-cf84-4965-8142-1998d0fd624a\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") "
Sep 15 12:53:08.807133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807045    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-socket\") pod \"3a424e6d-cf84-4965-8142-1998d0fd624a\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") "
Sep 15 12:53:08.807133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807072    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-envoy\") pod \"3a424e6d-cf84-4965-8142-1998d0fd624a\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") "
Sep 15 12:53:08.807133 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807093    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-credential-socket\") pod \"3a424e6d-cf84-4965-8142-1998d0fd624a\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") "
Sep 15 12:53:08.807395 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807155    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-token\") pod \"3a424e6d-cf84-4965-8142-1998d0fd624a\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") "
Sep 15 12:53:08.807395 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807193    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-certs\") pod \"3a424e6d-cf84-4965-8142-1998d0fd624a\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") "
Sep 15 12:53:08.807395 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807222    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a424e6d-cf84-4965-8142-1998d0fd624a-istiod-ca-cert\") pod \"3a424e6d-cf84-4965-8142-1998d0fd624a\" (UID: \"3a424e6d-cf84-4965-8142-1998d0fd624a\") "
Sep 15 12:53:08.807395 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807342    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a424e6d-cf84-4965-8142-1998d0fd624a" (UID: "3a424e6d-cf84-4965-8142-1998d0fd624a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:08.807395 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807377    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a424e6d-cf84-4965-8142-1998d0fd624a" (UID: "3a424e6d-cf84-4965-8142-1998d0fd624a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:08.807660 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807387    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a424e6d-cf84-4965-8142-1998d0fd624a" (UID: "3a424e6d-cf84-4965-8142-1998d0fd624a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:08.807660 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807589    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a424e6d-cf84-4965-8142-1998d0fd624a" (UID: "3a424e6d-cf84-4965-8142-1998d0fd624a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:08.807660 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807647    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:08.807831 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807663    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:08.807831 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807673    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:08.807831 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.807696    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a424e6d-cf84-4965-8142-1998d0fd624a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a424e6d-cf84-4965-8142-1998d0fd624a" (UID: "3a424e6d-cf84-4965-8142-1998d0fd624a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:53:08.809253 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.809230    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a424e6d-cf84-4965-8142-1998d0fd624a" (UID: "3a424e6d-cf84-4965-8142-1998d0fd624a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:08.809426 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.809411    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a424e6d-cf84-4965-8142-1998d0fd624a" (UID: "3a424e6d-cf84-4965-8142-1998d0fd624a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:53:08.809649 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.809634    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a424e6d-cf84-4965-8142-1998d0fd624a" (UID: "3a424e6d-cf84-4965-8142-1998d0fd624a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:53:08.809649 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.809640    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-kube-api-access-9jgfq" (OuterVolumeSpecName: "kube-api-access-9jgfq") pod "3a424e6d-cf84-4965-8142-1998d0fd624a" (UID: "3a424e6d-cf84-4965-8142-1998d0fd624a"). InnerVolumeSpecName "kube-api-access-9jgfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:53:08.908712 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.908661    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:08.908712 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.908707    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jgfq\" (UniqueName: \"kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-kube-api-access-9jgfq\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:08.908712 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.908722    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:08.908985 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.908732    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a424e6d-cf84-4965-8142-1998d0fd624a-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:08.908985 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.908740    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a424e6d-cf84-4965-8142-1998d0fd624a-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:08.908985 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:08.908747    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a424e6d-cf84-4965-8142-1998d0fd624a-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:09.044994 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:09.044951    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"]
Sep 15 12:53:09.048598 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:09.048571    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d9e7a1-openshift-default-7568d4b4d6-cdkhd"]
Sep 15 12:53:10.653748 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:10.653714    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" path="/var/lib/kubelet/pods/3a424e6d-cf84-4965-8142-1998d0fd624a/volumes"
Sep 15 12:53:36.830914 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:36.830875    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"]
Sep 15 12:53:47.436103 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.436067    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"]
Sep 15 12:53:47.436505 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.436302    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f" podUID="25994587-5d81-4602-8242-29fb1e013160" containerName="maas-api" containerID="cri-o://3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c" gracePeriod=30
Sep 15 12:53:47.692933 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.692867    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:53:47.802971 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.802935    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-metrics-certs\") pod \"25994587-5d81-4602-8242-29fb1e013160\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") "
Sep 15 12:53:47.803165 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.803061    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25994587-5d81-4602-8242-29fb1e013160-openshift-service-ca\") pod \"25994587-5d81-4602-8242-29fb1e013160\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") "
Sep 15 12:53:47.803165 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.803139    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-maas-api-tls\") pod \"25994587-5d81-4602-8242-29fb1e013160\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") "
Sep 15 12:53:47.803283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.803164    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6dr8\" (UniqueName: \"kubernetes.io/projected/25994587-5d81-4602-8242-29fb1e013160-kube-api-access-v6dr8\") pod \"25994587-5d81-4602-8242-29fb1e013160\" (UID: \"25994587-5d81-4602-8242-29fb1e013160\") "
Sep 15 12:53:47.804280 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.803586    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25994587-5d81-4602-8242-29fb1e013160-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "25994587-5d81-4602-8242-29fb1e013160" (UID: "25994587-5d81-4602-8242-29fb1e013160"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:53:47.805908 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.805881    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "25994587-5d81-4602-8242-29fb1e013160" (UID: "25994587-5d81-4602-8242-29fb1e013160"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:53:47.806046 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.805992    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25994587-5d81-4602-8242-29fb1e013160-kube-api-access-v6dr8" (OuterVolumeSpecName: "kube-api-access-v6dr8") pod "25994587-5d81-4602-8242-29fb1e013160" (UID: "25994587-5d81-4602-8242-29fb1e013160"). InnerVolumeSpecName "kube-api-access-v6dr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:53:47.806606 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.806585    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "25994587-5d81-4602-8242-29fb1e013160" (UID: "25994587-5d81-4602-8242-29fb1e013160"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:53:47.904227 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.904187    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25994587-5d81-4602-8242-29fb1e013160-openshift-service-ca\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:47.904227 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.904226    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-maas-api-tls\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:47.904482 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.904241    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6dr8\" (UniqueName: \"kubernetes.io/projected/25994587-5d81-4602-8242-29fb1e013160-kube-api-access-v6dr8\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:47.904482 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.904255    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25994587-5d81-4602-8242-29fb1e013160-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:47.939547 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.939517    2586 generic.go:356] "Generic (PLEG): container finished" podID="25994587-5d81-4602-8242-29fb1e013160" containerID="3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c" exitCode=0
Sep 15 12:53:47.939718 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.939561    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"
Sep 15 12:53:47.939718 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.939606    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f" event={"ID":"25994587-5d81-4602-8242-29fb1e013160","Type":"ContainerDied","Data":"3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c"}
Sep 15 12:53:47.939718 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.939661    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f" event={"ID":"25994587-5d81-4602-8242-29fb1e013160","Type":"ContainerDied","Data":"ff527fc91dd3c8a0b737b344c9a0a4c69487c3f9f5f05a8ea604b1aa91477c28"}
Sep 15 12:53:47.939718 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.939684    2586 scope.go:117] "RemoveContainer" containerID="3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c"
Sep 15 12:53:47.949455 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.949440    2586 scope.go:117] "RemoveContainer" containerID="3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c"
Sep 15 12:53:47.949719 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:53:47.949701    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c\": container with ID starting with 3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c not found: ID does not exist" containerID="3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c"
Sep 15 12:53:47.949839 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.949728    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c"} err="failed to get container status \"3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c\": rpc error: code = NotFound desc = could not find container \"3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c\": container with ID starting with 3530c63f03a9b3eb6dd63550d84528d2feccd7c11e3455e758f9cdc2cb65ef5c not found: ID does not exist"
Sep 15 12:53:47.961077 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.961056    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"]
Sep 15 12:53:47.964699 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:47.964681    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-11f865-6b78d8b5d-6pp9f"]
Sep 15 12:53:48.658948 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:48.658916    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25994587-5d81-4602-8242-29fb1e013160" path="/var/lib/kubelet/pods/25994587-5d81-4602-8242-29fb1e013160/volumes"
Sep 15 12:53:51.852434 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:51.852396    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh" podUID="81eb1888-e600-4fc5-95d3-798b6d21675d" containerName="main" containerID="cri-o://8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c" gracePeriod=45
Sep 15 12:53:52.122939 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.122911    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:53:52.252229 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252187    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-tmp-dir\") pod \"81eb1888-e600-4fc5-95d3-798b6d21675d\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") "
Sep 15 12:53:52.252419 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252254    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzwf8\" (UniqueName: \"kubernetes.io/projected/81eb1888-e600-4fc5-95d3-798b6d21675d-kube-api-access-rzwf8\") pod \"81eb1888-e600-4fc5-95d3-798b6d21675d\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") "
Sep 15 12:53:52.252419 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252299    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81eb1888-e600-4fc5-95d3-798b6d21675d-tls-certs\") pod \"81eb1888-e600-4fc5-95d3-798b6d21675d\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") "
Sep 15 12:53:52.252419 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252328    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-model-cache\") pod \"81eb1888-e600-4fc5-95d3-798b6d21675d\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") "
Sep 15 12:53:52.252419 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252406    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-dshm\") pod \"81eb1888-e600-4fc5-95d3-798b6d21675d\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") "
Sep 15 12:53:52.252661 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252455    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-home\") pod \"81eb1888-e600-4fc5-95d3-798b6d21675d\" (UID: \"81eb1888-e600-4fc5-95d3-798b6d21675d\") "
Sep 15 12:53:52.252661 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252465    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81eb1888-e600-4fc5-95d3-798b6d21675d" (UID: "81eb1888-e600-4fc5-95d3-798b6d21675d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:52.252661 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252618    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-model-cache" (OuterVolumeSpecName: "model-cache") pod "81eb1888-e600-4fc5-95d3-798b6d21675d" (UID: "81eb1888-e600-4fc5-95d3-798b6d21675d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:52.252863 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252833    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-home" (OuterVolumeSpecName: "home") pod "81eb1888-e600-4fc5-95d3-798b6d21675d" (UID: "81eb1888-e600-4fc5-95d3-798b6d21675d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:52.252931 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252881    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-tmp-dir\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:52.252931 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.252899    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-model-cache\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:52.255137 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.255106    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81eb1888-e600-4fc5-95d3-798b6d21675d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81eb1888-e600-4fc5-95d3-798b6d21675d" (UID: "81eb1888-e600-4fc5-95d3-798b6d21675d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:53:52.255278 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.255175    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-dshm" (OuterVolumeSpecName: "dshm") pod "81eb1888-e600-4fc5-95d3-798b6d21675d" (UID: "81eb1888-e600-4fc5-95d3-798b6d21675d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:52.255278 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.255180    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81eb1888-e600-4fc5-95d3-798b6d21675d-kube-api-access-rzwf8" (OuterVolumeSpecName: "kube-api-access-rzwf8") pod "81eb1888-e600-4fc5-95d3-798b6d21675d" (UID: "81eb1888-e600-4fc5-95d3-798b6d21675d"). InnerVolumeSpecName "kube-api-access-rzwf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:53:52.353520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.353485    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81eb1888-e600-4fc5-95d3-798b6d21675d-tls-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:52.353520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.353513    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-dshm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:52.353520 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.353522    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81eb1888-e600-4fc5-95d3-798b6d21675d-home\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:52.353741 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.353530    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzwf8\" (UniqueName: \"kubernetes.io/projected/81eb1888-e600-4fc5-95d3-798b6d21675d-kube-api-access-rzwf8\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:53:52.968828 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.968716    2586 generic.go:356] "Generic (PLEG): container finished" podID="81eb1888-e600-4fc5-95d3-798b6d21675d" containerID="8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c" exitCode=0
Sep 15 12:53:52.968828 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.968770    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"
Sep 15 12:53:52.968828 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.968800    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh" event={"ID":"81eb1888-e600-4fc5-95d3-798b6d21675d","Type":"ContainerDied","Data":"8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c"}
Sep 15 12:53:52.969711 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.968846    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh" event={"ID":"81eb1888-e600-4fc5-95d3-798b6d21675d","Type":"ContainerDied","Data":"7be7793f49d2871880f570ad91dda400477b7a926867569ac0c4b3f01a81d7b5"}
Sep 15 12:53:52.969711 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.968868    2586 scope.go:117] "RemoveContainer" containerID="8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c"
Sep 15 12:53:52.977870 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.977852    2586 scope.go:117] "RemoveContainer" containerID="8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c"
Sep 15 12:53:52.978151 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:53:52.978129    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c\": container with ID starting with 8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c not found: ID does not exist" containerID="8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c"
Sep 15 12:53:52.978220 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.978163    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c"} err="failed to get container status \"8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c\": rpc error: code = NotFound desc = could not find container \"8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c\": container with ID starting with 8c1ab3b23e231f84ad954d4f12d2494abd9524e503d375aa6c6d9112d9ee9f9c not found: ID does not exist"
Sep 15 12:53:52.986964 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.986940    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"]
Sep 15 12:53:52.993004 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:52.992980    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-11f865/embedding-w0-11f865-kserve-5dd66f6879-trkjh"]
Sep 15 12:53:54.652666 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:53:54.652633    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81eb1888-e600-4fc5-95d3-798b6d21675d" path="/var/lib/kubelet/pods/81eb1888-e600-4fc5-95d3-798b6d21675d/volumes"
Sep 15 12:54:07.853615 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:07.853578    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"]
Sep 15 12:54:18.510373 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.510324    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"]
Sep 15 12:54:18.510908 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.510596    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm" podUID="67e07c56-f32e-47b7-aadc-467bc900eb14" containerName="maas-api" containerID="cri-o://5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47" gracePeriod=30
Sep 15 12:54:18.768307 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.768235    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:54:18.826013 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.825981    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drwbv\" (UniqueName: \"kubernetes.io/projected/67e07c56-f32e-47b7-aadc-467bc900eb14-kube-api-access-drwbv\") pod \"67e07c56-f32e-47b7-aadc-467bc900eb14\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") "
Sep 15 12:54:18.826201 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.826049    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-metrics-certs\") pod \"67e07c56-f32e-47b7-aadc-467bc900eb14\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") "
Sep 15 12:54:18.826201 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.826069    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-maas-api-tls\") pod \"67e07c56-f32e-47b7-aadc-467bc900eb14\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") "
Sep 15 12:54:18.826201 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.826154    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67e07c56-f32e-47b7-aadc-467bc900eb14-openshift-service-ca\") pod \"67e07c56-f32e-47b7-aadc-467bc900eb14\" (UID: \"67e07c56-f32e-47b7-aadc-467bc900eb14\") "
Sep 15 12:54:18.826628 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.826588    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e07c56-f32e-47b7-aadc-467bc900eb14-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67e07c56-f32e-47b7-aadc-467bc900eb14" (UID: "67e07c56-f32e-47b7-aadc-467bc900eb14"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:54:18.828254 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.828230    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67e07c56-f32e-47b7-aadc-467bc900eb14" (UID: "67e07c56-f32e-47b7-aadc-467bc900eb14"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:54:18.828355 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.828323    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "67e07c56-f32e-47b7-aadc-467bc900eb14" (UID: "67e07c56-f32e-47b7-aadc-467bc900eb14"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:54:18.828395 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.828323    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e07c56-f32e-47b7-aadc-467bc900eb14-kube-api-access-drwbv" (OuterVolumeSpecName: "kube-api-access-drwbv") pod "67e07c56-f32e-47b7-aadc-467bc900eb14" (UID: "67e07c56-f32e-47b7-aadc-467bc900eb14"). InnerVolumeSpecName "kube-api-access-drwbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:54:18.927692 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.927655    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67e07c56-f32e-47b7-aadc-467bc900eb14-openshift-service-ca\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:18.927692 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.927689    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drwbv\" (UniqueName: \"kubernetes.io/projected/67e07c56-f32e-47b7-aadc-467bc900eb14-kube-api-access-drwbv\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:18.927989 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.927705    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:18.927989 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:18.927718    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67e07c56-f32e-47b7-aadc-467bc900eb14-maas-api-tls\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:19.127465 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:19.127378    2586 generic.go:356] "Generic (PLEG): container finished" podID="67e07c56-f32e-47b7-aadc-467bc900eb14" containerID="5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47" exitCode=0
Sep 15 12:54:19.127465 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:19.127437    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"
Sep 15 12:54:19.127693 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:19.127463    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm" event={"ID":"67e07c56-f32e-47b7-aadc-467bc900eb14","Type":"ContainerDied","Data":"5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47"}
Sep 15 12:54:19.127693 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:19.127546    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm" event={"ID":"67e07c56-f32e-47b7-aadc-467bc900eb14","Type":"ContainerDied","Data":"c8f9027f7e5fae3b244bcc860a7d39202136df5cc3d327a51d53ed88c5f2053e"}
Sep 15 12:54:19.127693 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:19.127575    2586 scope.go:117] "RemoveContainer" containerID="5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47"
Sep 15 12:54:19.137680 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:19.137662    2586 scope.go:117] "RemoveContainer" containerID="5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47"
Sep 15 12:54:19.137976 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:54:19.137953    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47\": container with ID starting with 5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47 not found: ID does not exist" containerID="5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47"
Sep 15 12:54:19.138062 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:19.137983    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47"} err="failed to get container status \"5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47\": rpc error: code = NotFound desc = could not find container \"5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47\": container with ID starting with 5f62a08f88ab6c76de30418ee48077a4c4e27ed10fc996ebb293dec4a7248b47 not found: ID does not exist"
Sep 15 12:54:19.148519 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:19.148496    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"]
Sep 15 12:54:19.152526 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:19.152494    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ee96ec-69c68cf9c8-875nm"]
Sep 15 12:54:20.652637 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:20.652603    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67e07c56-f32e-47b7-aadc-467bc900eb14" path="/var/lib/kubelet/pods/67e07c56-f32e-47b7-aadc-467bc900eb14/volumes"
Sep 15 12:54:22.873483 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:22.873442    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6" podUID="fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" containerName="main" containerID="cri-o://be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e" gracePeriod=45
Sep 15 12:54:23.136458 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.136396    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:54:23.152395 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.152357    2586 generic.go:356] "Generic (PLEG): container finished" podID="fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" containerID="be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e" exitCode=0
Sep 15 12:54:23.152607 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.152404    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"
Sep 15 12:54:23.152607 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.152460    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6" event={"ID":"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d","Type":"ContainerDied","Data":"be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e"}
Sep 15 12:54:23.152607 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.152495    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6" event={"ID":"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d","Type":"ContainerDied","Data":"126cdee56bddbfd65ecb3f0dbb657aa7df444043305ed9fba55e8d3f67e17ab8"}
Sep 15 12:54:23.152607 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.152516    2586 scope.go:117] "RemoveContainer" containerID="be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e"
Sep 15 12:54:23.165408 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.165382    2586 scope.go:117] "RemoveContainer" containerID="be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e"
Sep 15 12:54:23.165804 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:54:23.165768    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e\": container with ID starting with be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e not found: ID does not exist" containerID="be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e"
Sep 15 12:54:23.165928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.165817    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e"} err="failed to get container status \"be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e\": rpc error: code = NotFound desc = could not find container \"be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e\": container with ID starting with be99fb4428df3d93cc70b1bf8dc8f64035ec1f350273f7731ba42bdbc5bc885e not found: ID does not exist"
Sep 15 12:54:23.269759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.269725    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tmp-dir\") pod \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") "
Sep 15 12:54:23.269959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.269812    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6mzc\" (UniqueName: \"kubernetes.io/projected/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-kube-api-access-g6mzc\") pod \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") "
Sep 15 12:54:23.269959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.269856    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-dshm\") pod \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") "
Sep 15 12:54:23.269959 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.269884    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-home\") pod \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") "
Sep 15 12:54:23.270134 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.269984    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-model-cache\") pod \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") "
Sep 15 12:54:23.270134 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.270026    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tls-certs\") pod \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\" (UID: \"fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d\") "
Sep 15 12:54:23.270134 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.270095    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" (UID: "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:23.270315 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.270242    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-home" (OuterVolumeSpecName: "home") pod "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" (UID: "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:23.270315 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.270256    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-model-cache" (OuterVolumeSpecName: "model-cache") pod "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" (UID: "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:23.270445 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.270428    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tmp-dir\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:23.270515 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.270447    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-home\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:23.270515 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.270457    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-model-cache\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:23.272196 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.272171    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-dshm" (OuterVolumeSpecName: "dshm") pod "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" (UID: "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:23.272312 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.272177    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" (UID: "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:54:23.272364 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.272350    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-kube-api-access-g6mzc" (OuterVolumeSpecName: "kube-api-access-g6mzc") pod "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" (UID: "fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d"). InnerVolumeSpecName "kube-api-access-g6mzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:54:23.371838 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.371799    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-tls-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:23.371838 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.371839    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6mzc\" (UniqueName: \"kubernetes.io/projected/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-kube-api-access-g6mzc\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:23.372049 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.371853    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d-dshm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:23.474393 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.474359    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"]
Sep 15 12:54:23.479486 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:23.479460    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-ee96ec/distinct-2-w4-ee96ec-kserve-6d678b6d69-b2hs6"]
Sep 15 12:54:24.653038 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:24.653004    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" path="/var/lib/kubelet/pods/fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d/volumes"
Sep 15 12:54:25.020910 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:25.020869    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"]
Sep 15 12:54:25.021245 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:25.021215    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy" containerID="cri-o://80220efd626e953268da06f4f1ee37f4df06417f49f73567b24adabeffbd2c2b" gracePeriod=30
Sep 15 12:54:26.277238 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:26.277199    2586 patch_prober.go:28] interesting pod/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 12:54:26.277647 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:26.277260    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 12:54:30.195334 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.195113    2586 generic.go:356] "Generic (PLEG): container finished" podID="70eca63a-d3d3-480f-a771-9e59aedee860" containerID="80220efd626e953268da06f4f1ee37f4df06417f49f73567b24adabeffbd2c2b" exitCode=0
Sep 15 12:54:30.195334 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.195170    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" event={"ID":"70eca63a-d3d3-480f-a771-9e59aedee860","Type":"ContainerDied","Data":"80220efd626e953268da06f4f1ee37f4df06417f49f73567b24adabeffbd2c2b"}
Sep 15 12:54:30.195334 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.195227    2586 scope.go:117] "RemoveContainer" containerID="a0ae41559f2523cdfd14cef642386c22a7294b808de4eca296494dec7d7112ca"
Sep 15 12:54:30.274333 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.274308    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:54:30.345533 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345447    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-964fw\" (UniqueName: \"kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-kube-api-access-964fw\") pod \"70eca63a-d3d3-480f-a771-9e59aedee860\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") "
Sep 15 12:54:30.345533 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345516    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-data\") pod \"70eca63a-d3d3-480f-a771-9e59aedee860\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") "
Sep 15 12:54:30.345759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345571    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-istio-token\") pod \"70eca63a-d3d3-480f-a771-9e59aedee860\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") "
Sep 15 12:54:30.345759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345604    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70eca63a-d3d3-480f-a771-9e59aedee860-istiod-ca-cert\") pod \"70eca63a-d3d3-480f-a771-9e59aedee860\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") "
Sep 15 12:54:30.345759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345663    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-socket\") pod \"70eca63a-d3d3-480f-a771-9e59aedee860\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") "
Sep 15 12:54:30.345759 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345691    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-certs\") pod \"70eca63a-d3d3-480f-a771-9e59aedee860\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") "
Sep 15 12:54:30.345999 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345767    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70eca63a-d3d3-480f-a771-9e59aedee860-istio-podinfo\") pod \"70eca63a-d3d3-480f-a771-9e59aedee860\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") "
Sep 15 12:54:30.345999 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345814    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-credential-socket\") pod \"70eca63a-d3d3-480f-a771-9e59aedee860\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") "
Sep 15 12:54:30.345999 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345871    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-envoy\") pod \"70eca63a-d3d3-480f-a771-9e59aedee860\" (UID: \"70eca63a-d3d3-480f-a771-9e59aedee860\") "
Sep 15 12:54:30.345999 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345867    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-data" (OuterVolumeSpecName: "istio-data") pod "70eca63a-d3d3-480f-a771-9e59aedee860" (UID: "70eca63a-d3d3-480f-a771-9e59aedee860"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:30.345999 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.345942    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "70eca63a-d3d3-480f-a771-9e59aedee860" (UID: "70eca63a-d3d3-480f-a771-9e59aedee860"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:30.346257 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.346006    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "70eca63a-d3d3-480f-a771-9e59aedee860" (UID: "70eca63a-d3d3-480f-a771-9e59aedee860"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:30.346257 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.346051    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70eca63a-d3d3-480f-a771-9e59aedee860-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "70eca63a-d3d3-480f-a771-9e59aedee860" (UID: "70eca63a-d3d3-480f-a771-9e59aedee860"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:54:30.346257 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.346205    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "70eca63a-d3d3-480f-a771-9e59aedee860" (UID: "70eca63a-d3d3-480f-a771-9e59aedee860"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:30.346428 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.346324    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:30.346428 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.346341    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70eca63a-d3d3-480f-a771-9e59aedee860-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:30.346428 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.346354    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:30.346428 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.346366    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:30.346428 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.346379    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:30.347705 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.347679    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-kube-api-access-964fw" (OuterVolumeSpecName: "kube-api-access-964fw") pod "70eca63a-d3d3-480f-a771-9e59aedee860" (UID: "70eca63a-d3d3-480f-a771-9e59aedee860"). InnerVolumeSpecName "kube-api-access-964fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:54:30.347920 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.347890    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-istio-token" (OuterVolumeSpecName: "istio-token") pod "70eca63a-d3d3-480f-a771-9e59aedee860" (UID: "70eca63a-d3d3-480f-a771-9e59aedee860"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:54:30.348087 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.347937    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70eca63a-d3d3-480f-a771-9e59aedee860-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "70eca63a-d3d3-480f-a771-9e59aedee860" (UID: "70eca63a-d3d3-480f-a771-9e59aedee860"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:54:30.348087 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.347987    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "70eca63a-d3d3-480f-a771-9e59aedee860" (UID: "70eca63a-d3d3-480f-a771-9e59aedee860"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:30.447563 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.447529    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-964fw\" (UniqueName: \"kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-kube-api-access-964fw\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:30.447563 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.447563    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70eca63a-d3d3-480f-a771-9e59aedee860-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:30.447741 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.447578    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70eca63a-d3d3-480f-a771-9e59aedee860-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:30.447741 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:30.447591    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70eca63a-d3d3-480f-a771-9e59aedee860-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:31.202350 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:31.202316    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"
Sep 15 12:54:31.202868 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:31.202314    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv" event={"ID":"70eca63a-d3d3-480f-a771-9e59aedee860","Type":"ContainerDied","Data":"ab6782be5d63d2c0d9c38a574eb5c57b3c672f3f18c5d50a6ea9df71d8e6e041"}
Sep 15 12:54:31.202868 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:31.202453    2586 scope.go:117] "RemoveContainer" containerID="80220efd626e953268da06f4f1ee37f4df06417f49f73567b24adabeffbd2c2b"
Sep 15 12:54:31.219483 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:31.219458    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"]
Sep 15 12:54:31.223175 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:31.223152    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ee96ec-openshift-default-777bb9d8c8-7rlsv"]
Sep 15 12:54:32.653598 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:32.653564    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" path="/var/lib/kubelet/pods/70eca63a-d3d3-480f-a771-9e59aedee860/volumes"
Sep 15 12:54:40.332105 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:40.332068    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"]
Sep 15 12:54:55.351908 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.351849    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm" podUID="1720f4a8-a28e-4f97-af2c-8013bdbc4f13" containerName="main" containerID="cri-o://3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272" gracePeriod=45
Sep 15 12:54:55.607042 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.606978    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:54:55.702575 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.702530    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tls-certs\") pod \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") "
Sep 15 12:54:55.702806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.702586    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-home\") pod \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") "
Sep 15 12:54:55.702806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.702641    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgn5p\" (UniqueName: \"kubernetes.io/projected/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-kube-api-access-tgn5p\") pod \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") "
Sep 15 12:54:55.702806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.702715    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-model-cache\") pod \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") "
Sep 15 12:54:55.702806 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.702768    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-dshm\") pod \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") "
Sep 15 12:54:55.703029 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.702818    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tmp-dir\") pod \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\" (UID: \"1720f4a8-a28e-4f97-af2c-8013bdbc4f13\") "
Sep 15 12:54:55.703029 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.702937    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-home" (OuterVolumeSpecName: "home") pod "1720f4a8-a28e-4f97-af2c-8013bdbc4f13" (UID: "1720f4a8-a28e-4f97-af2c-8013bdbc4f13"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:55.703138 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.703022    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-model-cache" (OuterVolumeSpecName: "model-cache") pod "1720f4a8-a28e-4f97-af2c-8013bdbc4f13" (UID: "1720f4a8-a28e-4f97-af2c-8013bdbc4f13"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:55.703193 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.703146    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-model-cache\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:55.703193 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.703163    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-home\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:55.703193 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.703143    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1720f4a8-a28e-4f97-af2c-8013bdbc4f13" (UID: "1720f4a8-a28e-4f97-af2c-8013bdbc4f13"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:55.705196 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.705159    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-kube-api-access-tgn5p" (OuterVolumeSpecName: "kube-api-access-tgn5p") pod "1720f4a8-a28e-4f97-af2c-8013bdbc4f13" (UID: "1720f4a8-a28e-4f97-af2c-8013bdbc4f13"). InnerVolumeSpecName "kube-api-access-tgn5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:54:55.705455 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.705427    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1720f4a8-a28e-4f97-af2c-8013bdbc4f13" (UID: "1720f4a8-a28e-4f97-af2c-8013bdbc4f13"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:54:55.705561 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.705427    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-dshm" (OuterVolumeSpecName: "dshm") pod "1720f4a8-a28e-4f97-af2c-8013bdbc4f13" (UID: "1720f4a8-a28e-4f97-af2c-8013bdbc4f13"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:54:55.804190 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.804137    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgn5p\" (UniqueName: \"kubernetes.io/projected/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-kube-api-access-tgn5p\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:55.804190 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.804177    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-dshm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:55.804466 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.804212    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tmp-dir\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:55.804466 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:55.804221    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1720f4a8-a28e-4f97-af2c-8013bdbc4f13-tls-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:54:56.335322 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:56.335290    2586 generic.go:356] "Generic (PLEG): container finished" podID="1720f4a8-a28e-4f97-af2c-8013bdbc4f13" containerID="3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272" exitCode=0
Sep 15 12:54:56.335515 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:56.335331    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"
Sep 15 12:54:56.335515 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:56.335376    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm" event={"ID":"1720f4a8-a28e-4f97-af2c-8013bdbc4f13","Type":"ContainerDied","Data":"3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272"}
Sep 15 12:54:56.335515 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:56.335416    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm" event={"ID":"1720f4a8-a28e-4f97-af2c-8013bdbc4f13","Type":"ContainerDied","Data":"948e93e0169517f4dbf3e99ab5f6719629923ba37b61971d280d9e2cd47c4d8f"}
Sep 15 12:54:56.335515 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:56.335433    2586 scope.go:117] "RemoveContainer" containerID="3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272"
Sep 15 12:54:56.345172 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:56.345156    2586 scope.go:117] "RemoveContainer" containerID="3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272"
Sep 15 12:54:56.345469 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:54:56.345445    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272\": container with ID starting with 3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272 not found: ID does not exist" containerID="3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272"
Sep 15 12:54:56.345570 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:56.345475    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272"} err="failed to get container status \"3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272\": rpc error: code = NotFound desc = could not find container \"3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272\": container with ID starting with 3a76a7c335c272ac817a6a887cc006b0160b660ec5d05d0fb7fe1278b2795272 not found: ID does not exist"
Sep 15 12:54:56.356452 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:56.356428    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"]
Sep 15 12:54:56.360102 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:56.360081    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/auth-test-model-17f8f8-kserve-847d748dc9-7dwlm"]
Sep 15 12:54:56.653523 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:54:56.653446    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1720f4a8-a28e-4f97-af2c-8013bdbc4f13" path="/var/lib/kubelet/pods/1720f4a8-a28e-4f97-af2c-8013bdbc4f13/volumes"
Sep 15 12:55:50.652590 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.652559    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7710-5a25c1"]
Sep 15 12:55:50.653003 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.652933    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy"
Sep 15 12:55:50.653003 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.652946    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy"
Sep 15 12:55:50.653003 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.652954    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy"
Sep 15 12:55:50.653003 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.652960    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy"
Sep 15 12:55:50.653003 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.652980    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25994587-5d81-4602-8242-29fb1e013160" containerName="maas-api"
Sep 15 12:55:50.653003 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.652986    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="25994587-5d81-4602-8242-29fb1e013160" containerName="maas-api"
Sep 15 12:55:50.653003 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.652993    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e07c56-f32e-47b7-aadc-467bc900eb14" containerName="maas-api"
Sep 15 12:55:50.653003 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.652998    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e07c56-f32e-47b7-aadc-467bc900eb14" containerName="maas-api"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653009    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49204a0d-90a9-419d-9126-2b140892e0fd" containerName="payload-processing"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653014    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="49204a0d-90a9-419d-9126-2b140892e0fd" containerName="payload-processing"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653022    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1f2c109-17c9-4045-8e75-aaf57109caf1" containerName="payload-pre-processing"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653027    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1f2c109-17c9-4045-8e75-aaf57109caf1" containerName="payload-pre-processing"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653033    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1720f4a8-a28e-4f97-af2c-8013bdbc4f13" containerName="main"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653038    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1720f4a8-a28e-4f97-af2c-8013bdbc4f13" containerName="main"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653045    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81eb1888-e600-4fc5-95d3-798b6d21675d" containerName="main"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653049    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="81eb1888-e600-4fc5-95d3-798b6d21675d" containerName="main"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653056    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaa5c813-a6be-42dd-a3d6-2f33c78699f0" containerName="maas-api"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653061    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaa5c813-a6be-42dd-a3d6-2f33c78699f0" containerName="maas-api"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653067    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653073    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653081    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fafcfc94-94c1-45ee-a507-c26613679560" containerName="payload-processing"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653086    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="fafcfc94-94c1-45ee-a507-c26613679560" containerName="payload-processing"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653091    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" containerName="main"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653097    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" containerName="main"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653149    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653156    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e07c56-f32e-47b7-aadc-467bc900eb14" containerName="maas-api"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653161    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcdababb-fa59-4fb3-a4ce-e63e6ad76c9d" containerName="main"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653166    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="25994587-5d81-4602-8242-29fb1e013160" containerName="maas-api"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653172    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaa5c813-a6be-42dd-a3d6-2f33c78699f0" containerName="maas-api"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653177    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="81eb1888-e600-4fc5-95d3-798b6d21675d" containerName="main"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653183    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="49204a0d-90a9-419d-9126-2b140892e0fd" containerName="payload-processing"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653189    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="1720f4a8-a28e-4f97-af2c-8013bdbc4f13" containerName="main"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653193    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653198    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653202    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653207    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1f2c109-17c9-4045-8e75-aaf57109caf1" containerName="payload-pre-processing"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653212    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy"
Sep 15 12:55:50.653262 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.653217    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="fafcfc94-94c1-45ee-a507-c26613679560" containerName="payload-processing"
Sep 15 12:55:50.656295 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.656278    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7710-5a25c1"
Sep 15 12:55:50.658430 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.658408    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dhqjr\""
Sep 15 12:55:50.661928 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.661908    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7710-5a25c1"]
Sep 15 12:55:50.819531 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.819499    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv66k\" (UniqueName: \"kubernetes.io/projected/f175b96d-85ad-4bad-b089-a08ecd2469e7-kube-api-access-kv66k\") pod \"test-curl-7710-5a25c1\" (UID: \"f175b96d-85ad-4bad-b089-a08ecd2469e7\") " pod="opendatahub/test-curl-7710-5a25c1"
Sep 15 12:55:50.920647 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.920570    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv66k\" (UniqueName: \"kubernetes.io/projected/f175b96d-85ad-4bad-b089-a08ecd2469e7-kube-api-access-kv66k\") pod \"test-curl-7710-5a25c1\" (UID: \"f175b96d-85ad-4bad-b089-a08ecd2469e7\") " pod="opendatahub/test-curl-7710-5a25c1"
Sep 15 12:55:50.928686 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.928666    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv66k\" (UniqueName: \"kubernetes.io/projected/f175b96d-85ad-4bad-b089-a08ecd2469e7-kube-api-access-kv66k\") pod \"test-curl-7710-5a25c1\" (UID: \"f175b96d-85ad-4bad-b089-a08ecd2469e7\") " pod="opendatahub/test-curl-7710-5a25c1"
Sep 15 12:55:50.967296 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:50.967269    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7710-5a25c1"
Sep 15 12:55:51.233923 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:51.233898    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7710-5a25c1"]
Sep 15 12:55:51.236059 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:55:51.236026    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf175b96d_85ad_4bad_b089_a08ecd2469e7.slice/crio-c40c0dff944fbc83b5974a5f84adf5591ca119974c7e32ea13f8b2fae77a9b05 WatchSource:0}: Error finding container c40c0dff944fbc83b5974a5f84adf5591ca119974c7e32ea13f8b2fae77a9b05: Status 404 returned error can't find the container with id c40c0dff944fbc83b5974a5f84adf5591ca119974c7e32ea13f8b2fae77a9b05
Sep 15 12:55:51.238110 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:51.238091    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 12:55:51.609810 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:51.609714    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7710-5a25c1" event={"ID":"f175b96d-85ad-4bad-b089-a08ecd2469e7","Type":"ContainerStarted","Data":"c40c0dff944fbc83b5974a5f84adf5591ca119974c7e32ea13f8b2fae77a9b05"}
Sep 15 12:55:57.642602 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:57.642554    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7710-5a25c1" event={"ID":"f175b96d-85ad-4bad-b089-a08ecd2469e7","Type":"ContainerStarted","Data":"e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab"}
Sep 15 12:55:57.657088 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:57.657040    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7710-5a25c1" podStartSLOduration=1.876131103 podStartE2EDuration="7.657024925s" totalImagesPullingTime="5.780893822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:55:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:55:57.654564715 +0000 UTC m=+1825.466476158" watchObservedRunningTime="2026-09-15 12:55:57.657024925 +0000 UTC m=+1825.468936367"
Sep 15 12:55:58.282266 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:58.282231    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7710-5a25c1"]
Sep 15 12:55:58.286092 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:58.286063    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7710-5a25c1"]
Sep 15 12:55:58.647992 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:55:58.647965    2586 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7710-5a25c1" container="test-curl-7710-5a25c1"
Sep 15 12:55:59.652678 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:55:59.652643    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7710-5a25c1" podUID="f175b96d-85ad-4bad-b089-a08ecd2469e7" containerName="test-curl-7710-5a25c1" containerID="cri-o://e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab" gracePeriod=2
Sep 15 12:56:01.811320 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:01.811294    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7710-5a25c1"
Sep 15 12:56:01.927281 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:01.927202    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv66k\" (UniqueName: \"kubernetes.io/projected/f175b96d-85ad-4bad-b089-a08ecd2469e7-kube-api-access-kv66k\") pod \"f175b96d-85ad-4bad-b089-a08ecd2469e7\" (UID: \"f175b96d-85ad-4bad-b089-a08ecd2469e7\") "
Sep 15 12:56:01.929203 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:01.929181    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f175b96d-85ad-4bad-b089-a08ecd2469e7-kube-api-access-kv66k" (OuterVolumeSpecName: "kube-api-access-kv66k") pod "f175b96d-85ad-4bad-b089-a08ecd2469e7" (UID: "f175b96d-85ad-4bad-b089-a08ecd2469e7"). InnerVolumeSpecName "kube-api-access-kv66k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:56:02.028326 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:02.028294    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv66k\" (UniqueName: \"kubernetes.io/projected/f175b96d-85ad-4bad-b089-a08ecd2469e7-kube-api-access-kv66k\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:56:02.653082 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:02.653049    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f175b96d-85ad-4bad-b089-a08ecd2469e7" path="/var/lib/kubelet/pods/f175b96d-85ad-4bad-b089-a08ecd2469e7/volumes"
Sep 15 12:56:02.668087 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:02.668060    2586 generic.go:356] "Generic (PLEG): container finished" podID="f175b96d-85ad-4bad-b089-a08ecd2469e7" containerID="e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab" exitCode=137
Sep 15 12:56:02.668208 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:02.668096    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7710-5a25c1"
Sep 15 12:56:02.668208 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:02.668130    2586 scope.go:117] "RemoveContainer" containerID="e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab"
Sep 15 12:56:02.676390 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:02.676366    2586 scope.go:117] "RemoveContainer" containerID="e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab"
Sep 15 12:56:02.676656 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:56:02.676636    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab\": container with ID starting with e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab not found: ID does not exist" containerID="e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab"
Sep 15 12:56:02.676715 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:02.676664    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab"} err="failed to get container status \"e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab\": rpc error: code = NotFound desc = could not find container \"e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab\": container with ID starting with e759d51a5b74038b54ab54be6075257fe3ffd6b214635ac2fec15a40b51d18ab not found: ID does not exist"
Sep 15 12:56:47.355687 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.355651    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"]
Sep 15 12:56:47.356394 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.356365    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy"
Sep 15 12:56:47.356394 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.356392    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="863ff6d0-07ed-445a-8744-db60637b94eb" containerName="istio-proxy"
Sep 15 12:56:47.356627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.356423    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy"
Sep 15 12:56:47.356627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.356431    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy"
Sep 15 12:56:47.356627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.356443    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy"
Sep 15 12:56:47.356627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.356451    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="70eca63a-d3d3-480f-a771-9e59aedee860" containerName="istio-proxy"
Sep 15 12:56:47.356627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.356472    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f175b96d-85ad-4bad-b089-a08ecd2469e7" containerName="test-curl-7710-5a25c1"
Sep 15 12:56:47.356627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.356482    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f175b96d-85ad-4bad-b089-a08ecd2469e7" containerName="test-curl-7710-5a25c1"
Sep 15 12:56:47.356627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.356580    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a424e6d-cf84-4965-8142-1998d0fd624a" containerName="istio-proxy"
Sep 15 12:56:47.356627 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.356595    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f175b96d-85ad-4bad-b089-a08ecd2469e7" containerName="test-curl-7710-5a25c1"
Sep 15 12:56:47.359572 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.359552    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.362834 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.362814    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-17f8f8\"/\"rate-test-model-17f8f8-kserve-self-signed-certs\""
Sep 15 12:56:47.362950 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.362886    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-17f8f8\"/\"openshift-service-ca.crt\""
Sep 15 12:56:47.362950 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.362814    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-17f8f8\"/\"kube-root-ca.crt\""
Sep 15 12:56:47.362950 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.362818    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-17f8f8\"/\"default-dockercfg-jhmjz\""
Sep 15 12:56:47.368927 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.368902    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"]
Sep 15 12:56:47.461586 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.461549    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s28p6\" (UniqueName: \"kubernetes.io/projected/1fb04631-20bf-4a29-9b5d-e8819d29c643-kube-api-access-s28p6\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.461586 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.461594    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fb04631-20bf-4a29-9b5d-e8819d29c643-tls-certs\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.461846 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.461616    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-model-cache\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.461846 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.461713    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-home\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.461846 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.461736    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-tmp-dir\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.461846 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.461765    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-dshm\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.563286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.563243    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-home\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.563286 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.563293    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-tmp-dir\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.563567 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.563326    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-dshm\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.563567 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.563375    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s28p6\" (UniqueName: \"kubernetes.io/projected/1fb04631-20bf-4a29-9b5d-e8819d29c643-kube-api-access-s28p6\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.563567 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.563401    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fb04631-20bf-4a29-9b5d-e8819d29c643-tls-certs\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.563567 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.563418    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-model-cache\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.563862 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.563839    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-tmp-dir\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.563862 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.563849    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-home\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.563979 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.563950    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-model-cache\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.565766 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.565746    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-dshm\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.565942 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.565924    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fb04631-20bf-4a29-9b5d-e8819d29c643-tls-certs\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.570993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.570954    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s28p6\" (UniqueName: \"kubernetes.io/projected/1fb04631-20bf-4a29-9b5d-e8819d29c643-kube-api-access-s28p6\") pod \"rate-test-model-17f8f8-kserve-7c977548cf-h92dp\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") " pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.670758 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.670669    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:47.943536 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:47.943506    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"]
Sep 15 12:56:47.945531 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:56:47.945504    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fb04631_20bf_4a29_9b5d_e8819d29c643.slice/crio-0a20397ac610f765b188a5491c7c7a3e79eeda253bc4e5dc75bfdb779f23069e WatchSource:0}: Error finding container 0a20397ac610f765b188a5491c7c7a3e79eeda253bc4e5dc75bfdb779f23069e: Status 404 returned error can't find the container with id 0a20397ac610f765b188a5491c7c7a3e79eeda253bc4e5dc75bfdb779f23069e
Sep 15 12:56:48.903251 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:48.903217    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp" event={"ID":"1fb04631-20bf-4a29-9b5d-e8819d29c643","Type":"ContainerStarted","Data":"2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66"}
Sep 15 12:56:48.903642 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:48.903260    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp" event={"ID":"1fb04631-20bf-4a29-9b5d-e8819d29c643","Type":"ContainerStarted","Data":"0a20397ac610f765b188a5491c7c7a3e79eeda253bc4e5dc75bfdb779f23069e"}
Sep 15 12:56:48.922552 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:48.922500    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp" podStartSLOduration=1.9224847889999999 podStartE2EDuration="1.922484789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:56:48.92124344 +0000 UTC m=+1876.733154886" watchObservedRunningTime="2026-09-15 12:56:48.922484789 +0000 UTC m=+1876.734396230"
Sep 15 12:56:57.671896 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:57.671770    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:57.671896 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:57.671858    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:57.684933 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:57.684908    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:56:57.961600 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:56:57.961516    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:57:28.643576 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:28.643519    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"]
Sep 15 12:57:34.521854 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.521821    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"]
Sep 15 12:57:34.530028 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.529999    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.536608 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.533387    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f0171b-serving-cert\""
Sep 15 12:57:34.539535 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.539402    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"]
Sep 15 12:57:34.619364 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.619334    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-metrics-certs\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.619551 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.619389    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/751c8728-8c7e-4183-9e53-6a64f084cc1b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.619551 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.619454    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-625cc\" (UniqueName: \"kubernetes.io/projected/751c8728-8c7e-4183-9e53-6a64f084cc1b-kube-api-access-625cc\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.619636 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.619567    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.720072 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.720036    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.720248 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.720084    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-metrics-certs\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.720248 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.720124    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/751c8728-8c7e-4183-9e53-6a64f084cc1b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.720248 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.720149    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-625cc\" (UniqueName: \"kubernetes.io/projected/751c8728-8c7e-4183-9e53-6a64f084cc1b-kube-api-access-625cc\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.720886 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.720862    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/751c8728-8c7e-4183-9e53-6a64f084cc1b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.722501 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.722475    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-metrics-certs\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.722619 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.722570    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.728010 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.727992    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-625cc\" (UniqueName: \"kubernetes.io/projected/751c8728-8c7e-4183-9e53-6a64f084cc1b-kube-api-access-625cc\") pod \"maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:34.851144 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:34.851067    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:35.127284 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.127261    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"]
Sep 15 12:57:35.129078 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:57:35.129052    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751c8728_8c7e_4183_9e53_6a64f084cc1b.slice/crio-bac9ed62a26dcab82bc915fd792725ebad04dc74f50c2b985a8612c36cd37791 WatchSource:0}: Error finding container bac9ed62a26dcab82bc915fd792725ebad04dc74f50c2b985a8612c36cd37791: Status 404 returned error can't find the container with id bac9ed62a26dcab82bc915fd792725ebad04dc74f50c2b985a8612c36cd37791
Sep 15 12:57:35.140099 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.140075    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h" event={"ID":"751c8728-8c7e-4183-9e53-6a64f084cc1b","Type":"ContainerStarted","Data":"bac9ed62a26dcab82bc915fd792725ebad04dc74f50c2b985a8612c36cd37791"}
Sep 15 12:57:35.165348 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.165324    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"]
Sep 15 12:57:35.170244 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.170227    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 12:57:35.172438 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.172415    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f0171b\""
Sep 15 12:57:35.172553 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.172518    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f0171b-dockercfg-j67mc\""
Sep 15 12:57:35.176924 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.176902    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"]
Sep 15 12:57:35.226335 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.226300    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cc3bc70-6876-41a0-8307-53507f1e6db9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252\" (UID: \"2cc3bc70-6876-41a0-8307-53507f1e6db9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 12:57:35.226476 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.226345    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwml9\" (UniqueName: \"kubernetes.io/projected/2cc3bc70-6876-41a0-8307-53507f1e6db9-kube-api-access-hwml9\") pod \"payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252\" (UID: \"2cc3bc70-6876-41a0-8307-53507f1e6db9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 12:57:35.327993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.327963    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cc3bc70-6876-41a0-8307-53507f1e6db9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252\" (UID: \"2cc3bc70-6876-41a0-8307-53507f1e6db9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 12:57:35.328182 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.327998    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwml9\" (UniqueName: \"kubernetes.io/projected/2cc3bc70-6876-41a0-8307-53507f1e6db9-kube-api-access-hwml9\") pod \"payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252\" (UID: \"2cc3bc70-6876-41a0-8307-53507f1e6db9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 12:57:35.328743 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.328720    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cc3bc70-6876-41a0-8307-53507f1e6db9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252\" (UID: \"2cc3bc70-6876-41a0-8307-53507f1e6db9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 12:57:35.335702 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.335682    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwml9\" (UniqueName: \"kubernetes.io/projected/2cc3bc70-6876-41a0-8307-53507f1e6db9-kube-api-access-hwml9\") pod \"payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252\" (UID: \"2cc3bc70-6876-41a0-8307-53507f1e6db9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 12:57:35.484402 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.484376    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 12:57:35.758100 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:35.758071    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"]
Sep 15 12:57:35.760065 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:57:35.760038    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc3bc70_6876_41a0_8307_53507f1e6db9.slice/crio-cbdfbdf979d34f2ceaf065097977392caea0afb81c23e3e1ed45b19de564edee WatchSource:0}: Error finding container cbdfbdf979d34f2ceaf065097977392caea0afb81c23e3e1ed45b19de564edee: Status 404 returned error can't find the container with id cbdfbdf979d34f2ceaf065097977392caea0afb81c23e3e1ed45b19de564edee
Sep 15 12:57:36.147221 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:36.147120    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252" event={"ID":"2cc3bc70-6876-41a0-8307-53507f1e6db9","Type":"ContainerStarted","Data":"b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0"}
Sep 15 12:57:36.147221 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:36.147185    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252" event={"ID":"2cc3bc70-6876-41a0-8307-53507f1e6db9","Type":"ContainerStarted","Data":"cbdfbdf979d34f2ceaf065097977392caea0afb81c23e3e1ed45b19de564edee"}
Sep 15 12:57:36.147221 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:36.147213    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 12:57:36.163988 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:36.163941    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252" podStartSLOduration=1.163926736 podStartE2EDuration="1.163926736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:57:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:57:36.161647754 +0000 UTC m=+1923.973559189" watchObservedRunningTime="2026-09-15 12:57:36.163926736 +0000 UTC m=+1923.975838220"
Sep 15 12:57:38.159134 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:38.159100    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h" event={"ID":"751c8728-8c7e-4183-9e53-6a64f084cc1b","Type":"ContainerStarted","Data":"61b3abee2f383dd216fa9779dadad317bc0940d80d646f68714c88451701be35"}
Sep 15 12:57:38.159505 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:38.159197    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:38.176140 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:38.176092    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h" podStartSLOduration=1.9131489510000002 podStartE2EDuration="4.176079456s" totalImagesPullingTime="2.262930505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:57:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:57:38.173661385 +0000 UTC m=+1925.985572827" watchObservedRunningTime="2026-09-15 12:57:38.176079456 +0000 UTC m=+1925.987990898"
Sep 15 12:57:43.668249 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:43.668205    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp" podUID="1fb04631-20bf-4a29-9b5d-e8819d29c643" containerName="main" containerID="cri-o://2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66" gracePeriod=45
Sep 15 12:57:43.921445 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:43.921381    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:57:44.008726 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.008697    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-tmp-dir\") pod \"1fb04631-20bf-4a29-9b5d-e8819d29c643\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") "
Sep 15 12:57:44.008904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.008745    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-home\") pod \"1fb04631-20bf-4a29-9b5d-e8819d29c643\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") "
Sep 15 12:57:44.008904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.008832    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s28p6\" (UniqueName: \"kubernetes.io/projected/1fb04631-20bf-4a29-9b5d-e8819d29c643-kube-api-access-s28p6\") pod \"1fb04631-20bf-4a29-9b5d-e8819d29c643\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") "
Sep 15 12:57:44.008904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.008851    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fb04631-20bf-4a29-9b5d-e8819d29c643-tls-certs\") pod \"1fb04631-20bf-4a29-9b5d-e8819d29c643\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") "
Sep 15 12:57:44.008904 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.008896    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-model-cache\") pod \"1fb04631-20bf-4a29-9b5d-e8819d29c643\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") "
Sep 15 12:57:44.009107 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.008926    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-dshm\") pod \"1fb04631-20bf-4a29-9b5d-e8819d29c643\" (UID: \"1fb04631-20bf-4a29-9b5d-e8819d29c643\") "
Sep 15 12:57:44.009107 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.008989    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1fb04631-20bf-4a29-9b5d-e8819d29c643" (UID: "1fb04631-20bf-4a29-9b5d-e8819d29c643"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:57:44.009107 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.009099    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-home" (OuterVolumeSpecName: "home") pod "1fb04631-20bf-4a29-9b5d-e8819d29c643" (UID: "1fb04631-20bf-4a29-9b5d-e8819d29c643"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:57:44.009266 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.009207    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-model-cache" (OuterVolumeSpecName: "model-cache") pod "1fb04631-20bf-4a29-9b5d-e8819d29c643" (UID: "1fb04631-20bf-4a29-9b5d-e8819d29c643"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:57:44.009266 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.009255    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-tmp-dir\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:57:44.009340 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.009269    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-home\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:57:44.010992 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.010961    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-dshm" (OuterVolumeSpecName: "dshm") pod "1fb04631-20bf-4a29-9b5d-e8819d29c643" (UID: "1fb04631-20bf-4a29-9b5d-e8819d29c643"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:57:44.010992 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.010973    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fb04631-20bf-4a29-9b5d-e8819d29c643-kube-api-access-s28p6" (OuterVolumeSpecName: "kube-api-access-s28p6") pod "1fb04631-20bf-4a29-9b5d-e8819d29c643" (UID: "1fb04631-20bf-4a29-9b5d-e8819d29c643"). InnerVolumeSpecName "kube-api-access-s28p6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:57:44.011179 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.011077    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fb04631-20bf-4a29-9b5d-e8819d29c643-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1fb04631-20bf-4a29-9b5d-e8819d29c643" (UID: "1fb04631-20bf-4a29-9b5d-e8819d29c643"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:57:44.110619 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.110583    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s28p6\" (UniqueName: \"kubernetes.io/projected/1fb04631-20bf-4a29-9b5d-e8819d29c643-kube-api-access-s28p6\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:57:44.110619 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.110613    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fb04631-20bf-4a29-9b5d-e8819d29c643-tls-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:57:44.110845 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.110627    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-model-cache\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:57:44.110845 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.110639    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fb04631-20bf-4a29-9b5d-e8819d29c643-dshm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:57:44.170477 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.170446    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 12:57:44.191887 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.191816    2586 generic.go:356] "Generic (PLEG): container finished" podID="1fb04631-20bf-4a29-9b5d-e8819d29c643" containerID="2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66" exitCode=0
Sep 15 12:57:44.191887 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.191853    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"
Sep 15 12:57:44.192046 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.191900    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp" event={"ID":"1fb04631-20bf-4a29-9b5d-e8819d29c643","Type":"ContainerDied","Data":"2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66"}
Sep 15 12:57:44.192046 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.191936    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp" event={"ID":"1fb04631-20bf-4a29-9b5d-e8819d29c643","Type":"ContainerDied","Data":"0a20397ac610f765b188a5491c7c7a3e79eeda253bc4e5dc75bfdb779f23069e"}
Sep 15 12:57:44.192046 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.191952    2586 scope.go:117] "RemoveContainer" containerID="2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66"
Sep 15 12:57:44.204020 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.203997    2586 scope.go:117] "RemoveContainer" containerID="2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66"
Sep 15 12:57:44.204609 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:57:44.204417    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66\": container with ID starting with 2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66 not found: ID does not exist" containerID="2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66"
Sep 15 12:57:44.204609 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.204452    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66"} err="failed to get container status \"2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66\": rpc error: code = NotFound desc = could not find container \"2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66\": container with ID starting with 2f829fa7a9476f989d0d31f851c47eff7df8e5e7184c57ea3073262657147a66 not found: ID does not exist"
Sep 15 12:57:44.215547 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.215521    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"]
Sep 15 12:57:44.220287 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.220265    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-17f8f8/rate-test-model-17f8f8-kserve-7c977548cf-h92dp"]
Sep 15 12:57:44.653163 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:44.653132    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fb04631-20bf-4a29-9b5d-e8819d29c643" path="/var/lib/kubelet/pods/1fb04631-20bf-4a29-9b5d-e8819d29c643/volumes"
Sep 15 12:57:47.154546 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:57:47.154516    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 12:58:04.842730 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.842693    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"]
Sep 15 12:58:04.843203 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.843137    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fb04631-20bf-4a29-9b5d-e8819d29c643" containerName="main"
Sep 15 12:58:04.843203 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.843150    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fb04631-20bf-4a29-9b5d-e8819d29c643" containerName="main"
Sep 15 12:58:04.843283 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.843211    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fb04631-20bf-4a29-9b5d-e8819d29c643" containerName="main"
Sep 15 12:58:04.847625 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.847606    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 12:58:04.849796 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.849759    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-cd72a9\""
Sep 15 12:58:04.849953 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.849935    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-cd72a9-dockercfg-z7z8b\""
Sep 15 12:58:04.856372 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.856350    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"]
Sep 15 12:58:04.878970 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.878940    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"]
Sep 15 12:58:04.882577 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.882555    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 12:58:04.893159 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.893130    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"]
Sep 15 12:58:04.907449 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.907423    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mxq4\" (UniqueName: \"kubernetes.io/projected/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-kube-api-access-9mxq4\") pod \"payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727\" (UID: \"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 12:58:04.907989 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.907459    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49pkk\" (UniqueName: \"kubernetes.io/projected/6eb26d36-d1b2-4adc-96db-9283925a5643-kube-api-access-49pkk\") pod \"payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq\" (UID: \"6eb26d36-d1b2-4adc-96db-9283925a5643\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 12:58:04.908189 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.908168    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727\" (UID: \"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 12:58:04.908406 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:04.908389    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6eb26d36-d1b2-4adc-96db-9283925a5643-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq\" (UID: \"6eb26d36-d1b2-4adc-96db-9283925a5643\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 12:58:05.009503 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.009459    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mxq4\" (UniqueName: \"kubernetes.io/projected/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-kube-api-access-9mxq4\") pod \"payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727\" (UID: \"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 12:58:05.009668 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.009516    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49pkk\" (UniqueName: \"kubernetes.io/projected/6eb26d36-d1b2-4adc-96db-9283925a5643-kube-api-access-49pkk\") pod \"payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq\" (UID: \"6eb26d36-d1b2-4adc-96db-9283925a5643\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 12:58:05.009668 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.009543    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727\" (UID: \"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 12:58:05.009668 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.009641    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6eb26d36-d1b2-4adc-96db-9283925a5643-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq\" (UID: \"6eb26d36-d1b2-4adc-96db-9283925a5643\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 12:58:05.010359 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.010337    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727\" (UID: \"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 12:58:05.010565 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.010541    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6eb26d36-d1b2-4adc-96db-9283925a5643-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq\" (UID: \"6eb26d36-d1b2-4adc-96db-9283925a5643\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 12:58:05.017263 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.017234    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49pkk\" (UniqueName: \"kubernetes.io/projected/6eb26d36-d1b2-4adc-96db-9283925a5643-kube-api-access-49pkk\") pod \"payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq\" (UID: \"6eb26d36-d1b2-4adc-96db-9283925a5643\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 12:58:05.017457 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.017435    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mxq4\" (UniqueName: \"kubernetes.io/projected/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-kube-api-access-9mxq4\") pod \"payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727\" (UID: \"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 12:58:05.159458 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.159326    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 12:58:05.193890 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.193859    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 12:58:05.435949 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.435918    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"]
Sep 15 12:58:05.438367 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:58:05.438332    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb26d36_d1b2_4adc_96db_9283925a5643.slice/crio-893954bbb2b4980562a1cbf315dce5d9c29692969ce683ed9112a693799e7c48 WatchSource:0}: Error finding container 893954bbb2b4980562a1cbf315dce5d9c29692969ce683ed9112a693799e7c48: Status 404 returned error can't find the container with id 893954bbb2b4980562a1cbf315dce5d9c29692969ce683ed9112a693799e7c48
Sep 15 12:58:05.472036 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:05.472011    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"]
Sep 15 12:58:05.473465 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:58:05.473444    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac415e7_2c6c_4679_86bb_e2536f4ef3e6.slice/crio-b2412299a63704ed64ecdcb74dd2f8a349f7dd4784568f04025b960bc08c4358 WatchSource:0}: Error finding container b2412299a63704ed64ecdcb74dd2f8a349f7dd4784568f04025b960bc08c4358: Status 404 returned error can't find the container with id b2412299a63704ed64ecdcb74dd2f8a349f7dd4784568f04025b960bc08c4358
Sep 15 12:58:06.312823 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:06.312766    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq" event={"ID":"6eb26d36-d1b2-4adc-96db-9283925a5643","Type":"ContainerStarted","Data":"8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5"}
Sep 15 12:58:06.313334 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:06.312854    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq" event={"ID":"6eb26d36-d1b2-4adc-96db-9283925a5643","Type":"ContainerStarted","Data":"893954bbb2b4980562a1cbf315dce5d9c29692969ce683ed9112a693799e7c48"}
Sep 15 12:58:06.313334 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:06.312908    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 12:58:06.314562 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:06.314531    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727" event={"ID":"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6","Type":"ContainerStarted","Data":"9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e"}
Sep 15 12:58:06.314683 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:06.314588    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 12:58:06.314683 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:06.314606    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727" event={"ID":"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6","Type":"ContainerStarted","Data":"b2412299a63704ed64ecdcb74dd2f8a349f7dd4784568f04025b960bc08c4358"}
Sep 15 12:58:06.328835 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:06.328770    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq" podStartSLOduration=2.328750716 podStartE2EDuration="2.328750716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:58:06.32829772 +0000 UTC m=+1954.140209163" watchObservedRunningTime="2026-09-15 12:58:06.328750716 +0000 UTC m=+1954.140662160"
Sep 15 12:58:06.345883 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:06.345840    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727" podStartSLOduration=2.3458240679999998 podStartE2EDuration="2.345824068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:58:06.34489862 +0000 UTC m=+1954.156810060" watchObservedRunningTime="2026-09-15 12:58:06.345824068 +0000 UTC m=+1954.157735567"
Sep 15 12:58:17.322770 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:17.322735    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 12:58:17.323296 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:17.322833    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 12:58:32.556177 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:58:32.556144    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727_7ac415e7-2c6c-4679-86bb-e2536f4ef3e6/payload-processing/0.log"
Sep 15 12:59:23.710144 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:23.710034    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252_2cc3bc70-6876-41a0-8307-53507f1e6db9/payload-processing/0.log"
Sep 15 12:59:23.943027 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:23.942996    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727_7ac415e7-2c6c-4679-86bb-e2536f4ef3e6/payload-processing/0.log"
Sep 15 12:59:31.162335 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.162297    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"]
Sep 15 12:59:31.166092 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.166069    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:31.168849 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.168827    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-85cea9-dockercfg-ljxbj\""
Sep 15 12:59:31.169023 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.168852    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-85cea9\""
Sep 15 12:59:31.178312 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.178291    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"]
Sep 15 12:59:31.240993 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.240958    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xdzd\" (UniqueName: \"kubernetes.io/projected/6c2371c7-2225-4596-b43d-dac23652859b-kube-api-access-9xdzd\") pod \"payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx\" (UID: \"6c2371c7-2225-4596-b43d-dac23652859b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:31.241176 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.241097    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c2371c7-2225-4596-b43d-dac23652859b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx\" (UID: \"6c2371c7-2225-4596-b43d-dac23652859b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:31.342349 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.342310    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c2371c7-2225-4596-b43d-dac23652859b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx\" (UID: \"6c2371c7-2225-4596-b43d-dac23652859b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:31.342546 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.342370    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xdzd\" (UniqueName: \"kubernetes.io/projected/6c2371c7-2225-4596-b43d-dac23652859b-kube-api-access-9xdzd\") pod \"payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx\" (UID: \"6c2371c7-2225-4596-b43d-dac23652859b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:31.343150 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.343124    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c2371c7-2225-4596-b43d-dac23652859b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx\" (UID: \"6c2371c7-2225-4596-b43d-dac23652859b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:31.350269 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.350247    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xdzd\" (UniqueName: \"kubernetes.io/projected/6c2371c7-2225-4596-b43d-dac23652859b-kube-api-access-9xdzd\") pod \"payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx\" (UID: \"6c2371c7-2225-4596-b43d-dac23652859b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:31.478116 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.478019    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:31.754861 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:31.754831    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"]
Sep 15 12:59:31.756510 ip-10-0-141-20 kubenswrapper[2586]: W0915 12:59:31.756478    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2371c7_2225_4596_b43d_dac23652859b.slice/crio-f7f1587e83067e1e31a5cee21f25d99b48d81a5e6902b456d885bb072cd3a870 WatchSource:0}: Error finding container f7f1587e83067e1e31a5cee21f25d99b48d81a5e6902b456d885bb072cd3a870: Status 404 returned error can't find the container with id f7f1587e83067e1e31a5cee21f25d99b48d81a5e6902b456d885bb072cd3a870
Sep 15 12:59:32.767690 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:32.767654    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx" event={"ID":"6c2371c7-2225-4596-b43d-dac23652859b","Type":"ContainerStarted","Data":"60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e"}
Sep 15 12:59:32.768121 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:32.767708    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx" event={"ID":"6c2371c7-2225-4596-b43d-dac23652859b","Type":"ContainerStarted","Data":"f7f1587e83067e1e31a5cee21f25d99b48d81a5e6902b456d885bb072cd3a870"}
Sep 15 12:59:32.768121 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:32.767808    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:32.785045 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:32.784989    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx" podStartSLOduration=1.784963206 podStartE2EDuration="1.784963206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:59:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:59:32.782347902 +0000 UTC m=+2040.594259335" watchObservedRunningTime="2026-09-15 12:59:32.784963206 +0000 UTC m=+2040.596874650"
Sep 15 12:59:43.774565 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:43.774537    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:59.406837 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.406753    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"]
Sep 15 12:59:59.407277 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.407013    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx" podUID="6c2371c7-2225-4596-b43d-dac23652859b" containerName="payload-pre-processing" containerID="cri-o://60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e" gracePeriod=30
Sep 15 12:59:59.662442 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.662382    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:59.723659 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.723630    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c2371c7-2225-4596-b43d-dac23652859b-plugins-config-volume\") pod \"6c2371c7-2225-4596-b43d-dac23652859b\" (UID: \"6c2371c7-2225-4596-b43d-dac23652859b\") "
Sep 15 12:59:59.723854 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.723689    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xdzd\" (UniqueName: \"kubernetes.io/projected/6c2371c7-2225-4596-b43d-dac23652859b-kube-api-access-9xdzd\") pod \"6c2371c7-2225-4596-b43d-dac23652859b\" (UID: \"6c2371c7-2225-4596-b43d-dac23652859b\") "
Sep 15 12:59:59.724110 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.724086    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c2371c7-2225-4596-b43d-dac23652859b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6c2371c7-2225-4596-b43d-dac23652859b" (UID: "6c2371c7-2225-4596-b43d-dac23652859b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:59:59.725871 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.725848    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c2371c7-2225-4596-b43d-dac23652859b-kube-api-access-9xdzd" (OuterVolumeSpecName: "kube-api-access-9xdzd") pod "6c2371c7-2225-4596-b43d-dac23652859b" (UID: "6c2371c7-2225-4596-b43d-dac23652859b"). InnerVolumeSpecName "kube-api-access-9xdzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:59:59.824747 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.824714    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c2371c7-2225-4596-b43d-dac23652859b-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:59:59.824747 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.824744    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xdzd\" (UniqueName: \"kubernetes.io/projected/6c2371c7-2225-4596-b43d-dac23652859b-kube-api-access-9xdzd\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 12:59:59.911060 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.911025    2586 generic.go:356] "Generic (PLEG): container finished" podID="6c2371c7-2225-4596-b43d-dac23652859b" containerID="60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e" exitCode=0
Sep 15 12:59:59.911237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.911092    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"
Sep 15 12:59:59.911237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.911111    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx" event={"ID":"6c2371c7-2225-4596-b43d-dac23652859b","Type":"ContainerDied","Data":"60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e"}
Sep 15 12:59:59.911237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.911183    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx" event={"ID":"6c2371c7-2225-4596-b43d-dac23652859b","Type":"ContainerDied","Data":"f7f1587e83067e1e31a5cee21f25d99b48d81a5e6902b456d885bb072cd3a870"}
Sep 15 12:59:59.911237 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.911219    2586 scope.go:117] "RemoveContainer" containerID="60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e"
Sep 15 12:59:59.923706 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.923686    2586 scope.go:117] "RemoveContainer" containerID="60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e"
Sep 15 12:59:59.924010 ip-10-0-141-20 kubenswrapper[2586]: E0915 12:59:59.923990    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e\": container with ID starting with 60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e not found: ID does not exist" containerID="60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e"
Sep 15 12:59:59.924081 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.924018    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e"} err="failed to get container status \"60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e\": rpc error: code = NotFound desc = could not find container \"60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e\": container with ID starting with 60ecbff247d82f605ae7156dcc8f0c90f133200cc95b98a174a9948f7123d73e not found: ID does not exist"
Sep 15 12:59:59.934812 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.934789    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"]
Sep 15 12:59:59.942929 ip-10-0-141-20 kubenswrapper[2586]: I0915 12:59:59.942907    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85cea9-bd8588cc8-sddcx"]
Sep 15 13:00:00.653316 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:00.653275    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c2371c7-2225-4596-b43d-dac23652859b" path="/var/lib/kubelet/pods/6c2371c7-2225-4596-b43d-dac23652859b/volumes"
Sep 15 13:00:24.885259 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:24.885222    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"]
Sep 15 13:00:24.885915 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:24.885499    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h" podUID="751c8728-8c7e-4183-9e53-6a64f084cc1b" containerName="maas-api" containerID="cri-o://61b3abee2f383dd216fa9779dadad317bc0940d80d646f68714c88451701be35" gracePeriod=30
Sep 15 13:00:24.943221 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:24.943187    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"]
Sep 15 13:00:24.943455 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:24.943426    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252" podUID="2cc3bc70-6876-41a0-8307-53507f1e6db9" containerName="payload-processing" containerID="cri-o://b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0" gracePeriod=30
Sep 15 13:00:25.050864 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.050829    2586 generic.go:356] "Generic (PLEG): container finished" podID="751c8728-8c7e-4183-9e53-6a64f084cc1b" containerID="61b3abee2f383dd216fa9779dadad317bc0940d80d646f68714c88451701be35" exitCode=0
Sep 15 13:00:25.051038 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.050901    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h" event={"ID":"751c8728-8c7e-4183-9e53-6a64f084cc1b","Type":"ContainerDied","Data":"61b3abee2f383dd216fa9779dadad317bc0940d80d646f68714c88451701be35"}
Sep 15 13:00:25.255107 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.255071    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 13:00:25.274648 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.274620    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 13:00:25.365649 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.365616    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-metrics-certs\") pod \"751c8728-8c7e-4183-9e53-6a64f084cc1b\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") "
Sep 15 13:00:25.365859 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.365671    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-625cc\" (UniqueName: \"kubernetes.io/projected/751c8728-8c7e-4183-9e53-6a64f084cc1b-kube-api-access-625cc\") pod \"751c8728-8c7e-4183-9e53-6a64f084cc1b\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") "
Sep 15 13:00:25.365859 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.365708    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/751c8728-8c7e-4183-9e53-6a64f084cc1b-openshift-service-ca\") pod \"751c8728-8c7e-4183-9e53-6a64f084cc1b\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") "
Sep 15 13:00:25.366000 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.365905    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwml9\" (UniqueName: \"kubernetes.io/projected/2cc3bc70-6876-41a0-8307-53507f1e6db9-kube-api-access-hwml9\") pod \"2cc3bc70-6876-41a0-8307-53507f1e6db9\" (UID: \"2cc3bc70-6876-41a0-8307-53507f1e6db9\") "
Sep 15 13:00:25.366059 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.366004    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-maas-api-tls\") pod \"751c8728-8c7e-4183-9e53-6a64f084cc1b\" (UID: \"751c8728-8c7e-4183-9e53-6a64f084cc1b\") "
Sep 15 13:00:25.366059 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.366052    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cc3bc70-6876-41a0-8307-53507f1e6db9-plugins-config-volume\") pod \"2cc3bc70-6876-41a0-8307-53507f1e6db9\" (UID: \"2cc3bc70-6876-41a0-8307-53507f1e6db9\") "
Sep 15 13:00:25.366149 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.366066    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/751c8728-8c7e-4183-9e53-6a64f084cc1b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "751c8728-8c7e-4183-9e53-6a64f084cc1b" (UID: "751c8728-8c7e-4183-9e53-6a64f084cc1b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:00:25.366463 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.366429    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/751c8728-8c7e-4183-9e53-6a64f084cc1b-openshift-service-ca\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:00:25.366576 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.366466    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cc3bc70-6876-41a0-8307-53507f1e6db9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2cc3bc70-6876-41a0-8307-53507f1e6db9" (UID: "2cc3bc70-6876-41a0-8307-53507f1e6db9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:00:25.367964 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.367930    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "751c8728-8c7e-4183-9e53-6a64f084cc1b" (UID: "751c8728-8c7e-4183-9e53-6a64f084cc1b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:00:25.367964 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.367938    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cc3bc70-6876-41a0-8307-53507f1e6db9-kube-api-access-hwml9" (OuterVolumeSpecName: "kube-api-access-hwml9") pod "2cc3bc70-6876-41a0-8307-53507f1e6db9" (UID: "2cc3bc70-6876-41a0-8307-53507f1e6db9"). InnerVolumeSpecName "kube-api-access-hwml9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:25.368091 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.367972    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/751c8728-8c7e-4183-9e53-6a64f084cc1b-kube-api-access-625cc" (OuterVolumeSpecName: "kube-api-access-625cc") pod "751c8728-8c7e-4183-9e53-6a64f084cc1b" (UID: "751c8728-8c7e-4183-9e53-6a64f084cc1b"). InnerVolumeSpecName "kube-api-access-625cc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:25.368091 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.368047    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "751c8728-8c7e-4183-9e53-6a64f084cc1b" (UID: "751c8728-8c7e-4183-9e53-6a64f084cc1b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:00:25.467909 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.467828    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-625cc\" (UniqueName: \"kubernetes.io/projected/751c8728-8c7e-4183-9e53-6a64f084cc1b-kube-api-access-625cc\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:00:25.467909 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.467856    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwml9\" (UniqueName: \"kubernetes.io/projected/2cc3bc70-6876-41a0-8307-53507f1e6db9-kube-api-access-hwml9\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:00:25.467909 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.467865    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-maas-api-tls\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:00:25.467909 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.467874    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cc3bc70-6876-41a0-8307-53507f1e6db9-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:00:25.467909 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:25.467884    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/751c8728-8c7e-4183-9e53-6a64f084cc1b-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:00:26.056915 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.056873    2586 generic.go:356] "Generic (PLEG): container finished" podID="2cc3bc70-6876-41a0-8307-53507f1e6db9" containerID="b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0" exitCode=0
Sep 15 13:00:26.056915 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.056922    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"
Sep 15 13:00:26.057475 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.056961    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252" event={"ID":"2cc3bc70-6876-41a0-8307-53507f1e6db9","Type":"ContainerDied","Data":"b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0"}
Sep 15 13:00:26.057475 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.057034    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252" event={"ID":"2cc3bc70-6876-41a0-8307-53507f1e6db9","Type":"ContainerDied","Data":"cbdfbdf979d34f2ceaf065097977392caea0afb81c23e3e1ed45b19de564edee"}
Sep 15 13:00:26.057475 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.057057    2586 scope.go:117] "RemoveContainer" containerID="b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0"
Sep 15 13:00:26.058698 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.058667    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"
Sep 15 13:00:26.058698 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.058676    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h" event={"ID":"751c8728-8c7e-4183-9e53-6a64f084cc1b","Type":"ContainerDied","Data":"bac9ed62a26dcab82bc915fd792725ebad04dc74f50c2b985a8612c36cd37791"}
Sep 15 13:00:26.067432 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.067416    2586 scope.go:117] "RemoveContainer" containerID="b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0"
Sep 15 13:00:26.067722 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:00:26.067702    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0\": container with ID starting with b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0 not found: ID does not exist" containerID="b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0"
Sep 15 13:00:26.067848 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.067734    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0"} err="failed to get container status \"b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0\": rpc error: code = NotFound desc = could not find container \"b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0\": container with ID starting with b951d52d72d5682a73ac0b535543847158393e27abcafe13f14821bafa2ae2f0 not found: ID does not exist"
Sep 15 13:00:26.067848 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.067757    2586 scope.go:117] "RemoveContainer" containerID="61b3abee2f383dd216fa9779dadad317bc0940d80d646f68714c88451701be35"
Sep 15 13:00:26.078882 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.078859    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"]
Sep 15 13:00:26.082429 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.082407    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f0171b-56c4756475-kk252"]
Sep 15 13:00:26.092673 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.092650    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"]
Sep 15 13:00:26.096897 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.096872    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0171b-65dfcbdc8f-hfd7h"]
Sep 15 13:00:26.653383 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.653345    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cc3bc70-6876-41a0-8307-53507f1e6db9" path="/var/lib/kubelet/pods/2cc3bc70-6876-41a0-8307-53507f1e6db9/volumes"
Sep 15 13:00:26.653865 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:00:26.653845    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="751c8728-8c7e-4183-9e53-6a64f084cc1b" path="/var/lib/kubelet/pods/751c8728-8c7e-4183-9e53-6a64f084cc1b/volumes"
Sep 15 13:01:24.949388 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:24.949303    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"]
Sep 15 13:01:24.949885 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:24.949568    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727" podUID="7ac415e7-2c6c-4679-86bb-e2536f4ef3e6" containerName="payload-processing" containerID="cri-o://9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e" gracePeriod=30
Sep 15 13:01:24.959178 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:24.959150    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"]
Sep 15 13:01:24.959543 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:24.959510    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq" podUID="6eb26d36-d1b2-4adc-96db-9283925a5643" containerName="payload-pre-processing" containerID="cri-o://8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5" gracePeriod=30
Sep 15 13:01:25.221223 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.221189    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 13:01:25.251617 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.251593    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 13:01:25.322550 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.322520    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-plugins-config-volume\") pod \"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6\" (UID: \"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6\") "
Sep 15 13:01:25.322718 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.322618    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mxq4\" (UniqueName: \"kubernetes.io/projected/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-kube-api-access-9mxq4\") pod \"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6\" (UID: \"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6\") "
Sep 15 13:01:25.322983 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.322957    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ac415e7-2c6c-4679-86bb-e2536f4ef3e6" (UID: "7ac415e7-2c6c-4679-86bb-e2536f4ef3e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:01:25.324557 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.324533    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-kube-api-access-9mxq4" (OuterVolumeSpecName: "kube-api-access-9mxq4") pod "7ac415e7-2c6c-4679-86bb-e2536f4ef3e6" (UID: "7ac415e7-2c6c-4679-86bb-e2536f4ef3e6"). InnerVolumeSpecName "kube-api-access-9mxq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:01:25.368446 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.368419    2586 generic.go:356] "Generic (PLEG): container finished" podID="6eb26d36-d1b2-4adc-96db-9283925a5643" containerID="8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5" exitCode=0
Sep 15 13:01:25.368581 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.368469    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"
Sep 15 13:01:25.368581 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.368498    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq" event={"ID":"6eb26d36-d1b2-4adc-96db-9283925a5643","Type":"ContainerDied","Data":"8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5"}
Sep 15 13:01:25.368581 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.368555    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq" event={"ID":"6eb26d36-d1b2-4adc-96db-9283925a5643","Type":"ContainerDied","Data":"893954bbb2b4980562a1cbf315dce5d9c29692969ce683ed9112a693799e7c48"}
Sep 15 13:01:25.368581 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.368580    2586 scope.go:117] "RemoveContainer" containerID="8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5"
Sep 15 13:01:25.369960 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.369939    2586 generic.go:356] "Generic (PLEG): container finished" podID="7ac415e7-2c6c-4679-86bb-e2536f4ef3e6" containerID="9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e" exitCode=0
Sep 15 13:01:25.370067 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.369986    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"
Sep 15 13:01:25.370134 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.369984    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727" event={"ID":"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6","Type":"ContainerDied","Data":"9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e"}
Sep 15 13:01:25.370134 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.370114    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727" event={"ID":"7ac415e7-2c6c-4679-86bb-e2536f4ef3e6","Type":"ContainerDied","Data":"b2412299a63704ed64ecdcb74dd2f8a349f7dd4784568f04025b960bc08c4358"}
Sep 15 13:01:25.379157 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.378956    2586 scope.go:117] "RemoveContainer" containerID="8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5"
Sep 15 13:01:25.380169 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:01:25.380130    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5\": container with ID starting with 8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5 not found: ID does not exist" containerID="8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5"
Sep 15 13:01:25.380284 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.380178    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5"} err="failed to get container status \"8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5\": rpc error: code = NotFound desc = could not find container \"8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5\": container with ID starting with 8825c969e8b27373fae4565830315f0f5e2b5889dd18f9aed502f0c3934d19f5 not found: ID does not exist"
Sep 15 13:01:25.380284 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.380203    2586 scope.go:117] "RemoveContainer" containerID="9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e"
Sep 15 13:01:25.387563 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.387547    2586 scope.go:117] "RemoveContainer" containerID="9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e"
Sep 15 13:01:25.387851 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:01:25.387831    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e\": container with ID starting with 9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e not found: ID does not exist" containerID="9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e"
Sep 15 13:01:25.387921 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.387856    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e"} err="failed to get container status \"9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e\": rpc error: code = NotFound desc = could not find container \"9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e\": container with ID starting with 9c1b0630976683ef39115789a1c9a0188f2e5330890866f864686b90f0c3149e not found: ID does not exist"
Sep 15 13:01:25.393463 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.393444    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"]
Sep 15 13:01:25.395664 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.395639    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cd72a9-857d957575-jv727"]
Sep 15 13:01:25.423804 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.423770    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6eb26d36-d1b2-4adc-96db-9283925a5643-plugins-config-volume\") pod \"6eb26d36-d1b2-4adc-96db-9283925a5643\" (UID: \"6eb26d36-d1b2-4adc-96db-9283925a5643\") "
Sep 15 13:01:25.423936 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.423824    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49pkk\" (UniqueName: \"kubernetes.io/projected/6eb26d36-d1b2-4adc-96db-9283925a5643-kube-api-access-49pkk\") pod \"6eb26d36-d1b2-4adc-96db-9283925a5643\" (UID: \"6eb26d36-d1b2-4adc-96db-9283925a5643\") "
Sep 15 13:01:25.424051 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.424038    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:01:25.424125 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.424051    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mxq4\" (UniqueName: \"kubernetes.io/projected/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6-kube-api-access-9mxq4\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:01:25.424176 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.424127    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6eb26d36-d1b2-4adc-96db-9283925a5643-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6eb26d36-d1b2-4adc-96db-9283925a5643" (UID: "6eb26d36-d1b2-4adc-96db-9283925a5643"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:01:25.425720 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.425696    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6eb26d36-d1b2-4adc-96db-9283925a5643-kube-api-access-49pkk" (OuterVolumeSpecName: "kube-api-access-49pkk") pod "6eb26d36-d1b2-4adc-96db-9283925a5643" (UID: "6eb26d36-d1b2-4adc-96db-9283925a5643"). InnerVolumeSpecName "kube-api-access-49pkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:01:25.524897 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.524871    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6eb26d36-d1b2-4adc-96db-9283925a5643-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:01:25.524897 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.524894    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49pkk\" (UniqueName: \"kubernetes.io/projected/6eb26d36-d1b2-4adc-96db-9283925a5643-kube-api-access-49pkk\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:01:25.688809 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.688742    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"]
Sep 15 13:01:25.690701 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:25.690674    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cd72a9-666d59fbcd-fmrfq"]
Sep 15 13:01:26.652955 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:26.652913    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6eb26d36-d1b2-4adc-96db-9283925a5643" path="/var/lib/kubelet/pods/6eb26d36-d1b2-4adc-96db-9283925a5643/volumes"
Sep 15 13:01:26.653563 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:26.653541    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ac415e7-2c6c-4679-86bb-e2536f4ef3e6" path="/var/lib/kubelet/pods/7ac415e7-2c6c-4679-86bb-e2536f4ef3e6/volumes"
Sep 15 13:01:42.585073 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.585029    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"]
Sep 15 13:01:42.586655 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.586008    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7" podUID="89ec550d-4112-4b55-87b0-5318a1ade216" containerName="maas-api" containerID="cri-o://b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2" gracePeriod=30
Sep 15 13:01:42.655343 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.655316    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"]
Sep 15 13:01:42.655524 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.655503    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs" podUID="287fc015-568a-40cf-bb2a-0f4e46470858" containerName="payload-pre-processing" containerID="cri-o://89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00" gracePeriod=30
Sep 15 13:01:42.936413 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.936387    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 13:01:42.939133 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.939111    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 13:01:42.985587 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.985555    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-metrics-certs\") pod \"89ec550d-4112-4b55-87b0-5318a1ade216\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") "
Sep 15 13:01:42.985753 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.985630    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-maas-api-tls\") pod \"89ec550d-4112-4b55-87b0-5318a1ade216\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") "
Sep 15 13:01:42.985753 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.985699    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ec550d-4112-4b55-87b0-5318a1ade216-openshift-service-ca\") pod \"89ec550d-4112-4b55-87b0-5318a1ade216\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") "
Sep 15 13:01:42.985753 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.985723    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmb6p\" (UniqueName: \"kubernetes.io/projected/287fc015-568a-40cf-bb2a-0f4e46470858-kube-api-access-mmb6p\") pod \"287fc015-568a-40cf-bb2a-0f4e46470858\" (UID: \"287fc015-568a-40cf-bb2a-0f4e46470858\") "
Sep 15 13:01:42.985753 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.985740    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/287fc015-568a-40cf-bb2a-0f4e46470858-plugins-config-volume\") pod \"287fc015-568a-40cf-bb2a-0f4e46470858\" (UID: \"287fc015-568a-40cf-bb2a-0f4e46470858\") "
Sep 15 13:01:42.986025 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.985757    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j9rj\" (UniqueName: \"kubernetes.io/projected/89ec550d-4112-4b55-87b0-5318a1ade216-kube-api-access-5j9rj\") pod \"89ec550d-4112-4b55-87b0-5318a1ade216\" (UID: \"89ec550d-4112-4b55-87b0-5318a1ade216\") "
Sep 15 13:01:42.986151 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.986122    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89ec550d-4112-4b55-87b0-5318a1ade216-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "89ec550d-4112-4b55-87b0-5318a1ade216" (UID: "89ec550d-4112-4b55-87b0-5318a1ade216"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:01:42.986361 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.986335    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/287fc015-568a-40cf-bb2a-0f4e46470858-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "287fc015-568a-40cf-bb2a-0f4e46470858" (UID: "287fc015-568a-40cf-bb2a-0f4e46470858"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:01:42.987836 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.987816    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/287fc015-568a-40cf-bb2a-0f4e46470858-kube-api-access-mmb6p" (OuterVolumeSpecName: "kube-api-access-mmb6p") pod "287fc015-568a-40cf-bb2a-0f4e46470858" (UID: "287fc015-568a-40cf-bb2a-0f4e46470858"). InnerVolumeSpecName "kube-api-access-mmb6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:01:42.988201 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.988181    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "89ec550d-4112-4b55-87b0-5318a1ade216" (UID: "89ec550d-4112-4b55-87b0-5318a1ade216"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:01:42.988271 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.988199    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "89ec550d-4112-4b55-87b0-5318a1ade216" (UID: "89ec550d-4112-4b55-87b0-5318a1ade216"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:01:42.988436 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:42.988417    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ec550d-4112-4b55-87b0-5318a1ade216-kube-api-access-5j9rj" (OuterVolumeSpecName: "kube-api-access-5j9rj") pod "89ec550d-4112-4b55-87b0-5318a1ade216" (UID: "89ec550d-4112-4b55-87b0-5318a1ade216"). InnerVolumeSpecName "kube-api-access-5j9rj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:01:43.086592 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.086558    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ec550d-4112-4b55-87b0-5318a1ade216-openshift-service-ca\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:01:43.086592 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.086586    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmb6p\" (UniqueName: \"kubernetes.io/projected/287fc015-568a-40cf-bb2a-0f4e46470858-kube-api-access-mmb6p\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:01:43.086592 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.086597    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/287fc015-568a-40cf-bb2a-0f4e46470858-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:01:43.086859 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.086609    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j9rj\" (UniqueName: \"kubernetes.io/projected/89ec550d-4112-4b55-87b0-5318a1ade216-kube-api-access-5j9rj\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:01:43.086859 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.086618    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:01:43.086859 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.086626    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ec550d-4112-4b55-87b0-5318a1ade216-maas-api-tls\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:01:43.460736 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.460704    2586 generic.go:356] "Generic (PLEG): container finished" podID="89ec550d-4112-4b55-87b0-5318a1ade216" containerID="b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2" exitCode=0
Sep 15 13:01:43.460940 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.460748    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"
Sep 15 13:01:43.460940 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.460809    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7" event={"ID":"89ec550d-4112-4b55-87b0-5318a1ade216","Type":"ContainerDied","Data":"b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2"}
Sep 15 13:01:43.460940 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.460865    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7" event={"ID":"89ec550d-4112-4b55-87b0-5318a1ade216","Type":"ContainerDied","Data":"d221e59b9c57fa4435a4443f4951102b4ae66755533482abb25825878481ed1d"}
Sep 15 13:01:43.460940 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.460891    2586 scope.go:117] "RemoveContainer" containerID="b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2"
Sep 15 13:01:43.462199 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.462182    2586 generic.go:356] "Generic (PLEG): container finished" podID="287fc015-568a-40cf-bb2a-0f4e46470858" containerID="89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00" exitCode=0
Sep 15 13:01:43.462308 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.462242    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"
Sep 15 13:01:43.462308 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.462264    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs" event={"ID":"287fc015-568a-40cf-bb2a-0f4e46470858","Type":"ContainerDied","Data":"89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00"}
Sep 15 13:01:43.462422 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.462315    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs" event={"ID":"287fc015-568a-40cf-bb2a-0f4e46470858","Type":"ContainerDied","Data":"1ce5fff84bdced998f9b7dd2d85533dd34933fe885b6c28ed44bdc50f346ec18"}
Sep 15 13:01:43.471241 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.471163    2586 scope.go:117] "RemoveContainer" containerID="b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2"
Sep 15 13:01:43.471489 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:01:43.471470    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2\": container with ID starting with b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2 not found: ID does not exist" containerID="b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2"
Sep 15 13:01:43.471580 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.471502    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2"} err="failed to get container status \"b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2\": rpc error: code = NotFound desc = could not find container \"b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2\": container with ID starting with b7bc4c8e8d47afc00873012aea9f18582a255eeb1b3019de1473103369b517b2 not found: ID does not exist"
Sep 15 13:01:43.471580 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.471526    2586 scope.go:117] "RemoveContainer" containerID="89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00"
Sep 15 13:01:43.479521 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.479500    2586 scope.go:117] "RemoveContainer" containerID="89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00"
Sep 15 13:01:43.479817 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:01:43.479798    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00\": container with ID starting with 89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00 not found: ID does not exist" containerID="89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00"
Sep 15 13:01:43.479880 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.479824    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00"} err="failed to get container status \"89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00\": rpc error: code = NotFound desc = could not find container \"89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00\": container with ID starting with 89f5260f85ffc89a53057b97de08e183ac6666147259538280e4777648b1fe00 not found: ID does not exist"
Sep 15 13:01:43.485967 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.485945    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"]
Sep 15 13:01:43.489113 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.489091    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-29a9e9-8ffd84784-nvxs7"]
Sep 15 13:01:43.498762 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.498739    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"]
Sep 15 13:01:43.501893 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:43.501873    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-29a9e9-8986f857d-nbgxs"]
Sep 15 13:01:44.652534 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:44.652503    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="287fc015-568a-40cf-bb2a-0f4e46470858" path="/var/lib/kubelet/pods/287fc015-568a-40cf-bb2a-0f4e46470858/volumes"
Sep 15 13:01:44.652935 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:01:44.652925    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89ec550d-4112-4b55-87b0-5318a1ade216" path="/var/lib/kubelet/pods/89ec550d-4112-4b55-87b0-5318a1ade216/volumes"
Sep 15 13:02:00.152335 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.152295    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"]
Sep 15 13:02:00.155015 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.152559    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl" podUID="41954fc4-a5d6-45a1-bd0b-4b507d7e598f" containerName="payload-pre-processing" containerID="cri-o://e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7" gracePeriod=30
Sep 15 13:02:00.429275 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.429206    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 13:02:00.546181 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.546150    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6q7x\" (UniqueName: \"kubernetes.io/projected/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-kube-api-access-q6q7x\") pod \"41954fc4-a5d6-45a1-bd0b-4b507d7e598f\" (UID: \"41954fc4-a5d6-45a1-bd0b-4b507d7e598f\") "
Sep 15 13:02:00.546376 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.546231    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-plugins-config-volume\") pod \"41954fc4-a5d6-45a1-bd0b-4b507d7e598f\" (UID: \"41954fc4-a5d6-45a1-bd0b-4b507d7e598f\") "
Sep 15 13:02:00.546581 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.546548    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "41954fc4-a5d6-45a1-bd0b-4b507d7e598f" (UID: "41954fc4-a5d6-45a1-bd0b-4b507d7e598f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:02:00.548379 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.548354    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-kube-api-access-q6q7x" (OuterVolumeSpecName: "kube-api-access-q6q7x") pod "41954fc4-a5d6-45a1-bd0b-4b507d7e598f" (UID: "41954fc4-a5d6-45a1-bd0b-4b507d7e598f"). InnerVolumeSpecName "kube-api-access-q6q7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:02:00.551509 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.551481    2586 generic.go:356] "Generic (PLEG): container finished" podID="41954fc4-a5d6-45a1-bd0b-4b507d7e598f" containerID="e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7" exitCode=0
Sep 15 13:02:00.551646 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.551520    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl" event={"ID":"41954fc4-a5d6-45a1-bd0b-4b507d7e598f","Type":"ContainerDied","Data":"e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7"}
Sep 15 13:02:00.551646 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.551536    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"
Sep 15 13:02:00.551646 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.551567    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl" event={"ID":"41954fc4-a5d6-45a1-bd0b-4b507d7e598f","Type":"ContainerDied","Data":"84329b42f90ce800f24ab161575e43370f13f2bc3be7204dad81d8c801e8ff9f"}
Sep 15 13:02:00.551646 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.551598    2586 scope.go:117] "RemoveContainer" containerID="e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7"
Sep 15 13:02:00.567664 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.567639    2586 scope.go:117] "RemoveContainer" containerID="e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7"
Sep 15 13:02:00.567988 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:02:00.567967    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7\": container with ID starting with e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7 not found: ID does not exist" containerID="e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7"
Sep 15 13:02:00.568063 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.567997    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7"} err="failed to get container status \"e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7\": rpc error: code = NotFound desc = could not find container \"e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7\": container with ID starting with e2abdb4de2afb4f7c841f9733732a45302b1a5862f1e6379de07e59b358a91e7 not found: ID does not exist"
Sep 15 13:02:00.581504 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.581463    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"]
Sep 15 13:02:00.583530 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.583507    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-17f8f8-65cb777df4-9fjrl"]
Sep 15 13:02:00.647001 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.646967    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6q7x\" (UniqueName: \"kubernetes.io/projected/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-kube-api-access-q6q7x\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:00.647001 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.646991    2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41954fc4-a5d6-45a1-bd0b-4b507d7e598f-plugins-config-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:00.653416 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:00.653391    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41954fc4-a5d6-45a1-bd0b-4b507d7e598f" path="/var/lib/kubelet/pods/41954fc4-a5d6-45a1-bd0b-4b507d7e598f/volumes"
Sep 15 13:02:06.739522 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:06.739489    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"]
Sep 15 13:02:06.739998 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:06.739739    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy" containerID="cri-o://426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f" gracePeriod=30
Sep 15 13:02:11.918675 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:11.918642    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65f48575b5-qhgwt"]
Sep 15 13:02:11.919152 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:11.919102    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65f48575b5-qhgwt" podUID="99d65925-7457-4832-b0e5-8cb75689824f" containerName="manager" containerID="cri-o://9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1" gracePeriod=30
Sep 15 13:02:12.062988 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.062962    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 13:02:12.151599 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.151575    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-certs\") pod \"17735be7-24d6-49ae-8c6a-96ee853b89e2\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") "
Sep 15 13:02:12.151761 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.151607    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-token\") pod \"17735be7-24d6-49ae-8c6a-96ee853b89e2\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") "
Sep 15 13:02:12.151761 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.151631    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17735be7-24d6-49ae-8c6a-96ee853b89e2-istiod-ca-cert\") pod \"17735be7-24d6-49ae-8c6a-96ee853b89e2\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") "
Sep 15 13:02:12.151761 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.151649    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-podinfo\") pod \"17735be7-24d6-49ae-8c6a-96ee853b89e2\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") "
Sep 15 13:02:12.151761 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.151669    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24fmp\" (UniqueName: \"kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-kube-api-access-24fmp\") pod \"17735be7-24d6-49ae-8c6a-96ee853b89e2\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") "
Sep 15 13:02:12.151761 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.151743    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-socket\") pod \"17735be7-24d6-49ae-8c6a-96ee853b89e2\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") "
Sep 15 13:02:12.152060 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.151839    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-data\") pod \"17735be7-24d6-49ae-8c6a-96ee853b89e2\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") "
Sep 15 13:02:12.152060 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.151894    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-envoy\") pod \"17735be7-24d6-49ae-8c6a-96ee853b89e2\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") "
Sep 15 13:02:12.152060 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.151950    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-credential-socket\") pod \"17735be7-24d6-49ae-8c6a-96ee853b89e2\" (UID: \"17735be7-24d6-49ae-8c6a-96ee853b89e2\") "
Sep 15 13:02:12.152060 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.151924    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "17735be7-24d6-49ae-8c6a-96ee853b89e2" (UID: "17735be7-24d6-49ae-8c6a-96ee853b89e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:02:12.152272 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.152074    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "17735be7-24d6-49ae-8c6a-96ee853b89e2" (UID: "17735be7-24d6-49ae-8c6a-96ee853b89e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:02:12.152272 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.152175    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "17735be7-24d6-49ae-8c6a-96ee853b89e2" (UID: "17735be7-24d6-49ae-8c6a-96ee853b89e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:02:12.152272 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.152182    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17735be7-24d6-49ae-8c6a-96ee853b89e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "17735be7-24d6-49ae-8c6a-96ee853b89e2" (UID: "17735be7-24d6-49ae-8c6a-96ee853b89e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:02:12.152435 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.152305    2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17735be7-24d6-49ae-8c6a-96ee853b89e2-istiod-ca-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.152435 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.152309    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "17735be7-24d6-49ae-8c6a-96ee853b89e2" (UID: "17735be7-24d6-49ae-8c6a-96ee853b89e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:02:12.152435 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.152322    2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.152435 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.152335    2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-data\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.152435 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.152347    2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-workload-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.154707 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.154676    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "17735be7-24d6-49ae-8c6a-96ee853b89e2" (UID: "17735be7-24d6-49ae-8c6a-96ee853b89e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:02:12.154824 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.154729    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-kube-api-access-24fmp" (OuterVolumeSpecName: "kube-api-access-24fmp") pod "17735be7-24d6-49ae-8c6a-96ee853b89e2" (UID: "17735be7-24d6-49ae-8c6a-96ee853b89e2"). InnerVolumeSpecName "kube-api-access-24fmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:02:12.154824 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.154804    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "17735be7-24d6-49ae-8c6a-96ee853b89e2" (UID: "17735be7-24d6-49ae-8c6a-96ee853b89e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:02:12.154924 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.154804    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "17735be7-24d6-49ae-8c6a-96ee853b89e2" (UID: "17735be7-24d6-49ae-8c6a-96ee853b89e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:02:12.176300 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.176279    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 13:02:12.253649 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.253620    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-metrics-certs\") pod \"99d65925-7457-4832-b0e5-8cb75689824f\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") "
Sep 15 13:02:12.253849 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.253661    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dc5bf\" (UniqueName: \"kubernetes.io/projected/99d65925-7457-4832-b0e5-8cb75689824f-kube-api-access-dc5bf\") pod \"99d65925-7457-4832-b0e5-8cb75689824f\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") "
Sep 15 13:02:12.253849 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.253693    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-webhook-cert\") pod \"99d65925-7457-4832-b0e5-8cb75689824f\" (UID: \"99d65925-7457-4832-b0e5-8cb75689824f\") "
Sep 15 13:02:12.253997 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.253982    2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-token\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.254053 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.254000    2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-podinfo\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.254053 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.254011    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24fmp\" (UniqueName: \"kubernetes.io/projected/17735be7-24d6-49ae-8c6a-96ee853b89e2-kube-api-access-24fmp\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.254053 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.254030    2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-istio-envoy\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.254053 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.254045    2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17735be7-24d6-49ae-8c6a-96ee853b89e2-credential-socket\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.255736 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.255704    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d65925-7457-4832-b0e5-8cb75689824f-kube-api-access-dc5bf" (OuterVolumeSpecName: "kube-api-access-dc5bf") pod "99d65925-7457-4832-b0e5-8cb75689824f" (UID: "99d65925-7457-4832-b0e5-8cb75689824f"). InnerVolumeSpecName "kube-api-access-dc5bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:02:12.255890 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.255735    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "99d65925-7457-4832-b0e5-8cb75689824f" (UID: "99d65925-7457-4832-b0e5-8cb75689824f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:02:12.255988 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.255966    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "99d65925-7457-4832-b0e5-8cb75689824f" (UID: "99d65925-7457-4832-b0e5-8cb75689824f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:02:12.355398 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.355329    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.355398 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.355354    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dc5bf\" (UniqueName: \"kubernetes.io/projected/99d65925-7457-4832-b0e5-8cb75689824f-kube-api-access-dc5bf\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.355398 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.355364    2586 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99d65925-7457-4832-b0e5-8cb75689824f-webhook-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:02:12.616106 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.616030    2586 generic.go:356] "Generic (PLEG): container finished" podID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerID="426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f" exitCode=0
Sep 15 13:02:12.616106 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.616083    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"
Sep 15 13:02:12.616336 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.616148    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" event={"ID":"17735be7-24d6-49ae-8c6a-96ee853b89e2","Type":"ContainerDied","Data":"426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f"}
Sep 15 13:02:12.616336 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.616198    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662" event={"ID":"17735be7-24d6-49ae-8c6a-96ee853b89e2","Type":"ContainerDied","Data":"008b1ae56cda34faa9c3a7ea4b5166e437b3286c6252d55beb555ff266c31e31"}
Sep 15 13:02:12.616336 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.616221    2586 scope.go:117] "RemoveContainer" containerID="426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f"
Sep 15 13:02:12.617520 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.617493    2586 generic.go:356] "Generic (PLEG): container finished" podID="99d65925-7457-4832-b0e5-8cb75689824f" containerID="9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1" exitCode=0
Sep 15 13:02:12.617658 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.617522    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65f48575b5-qhgwt" event={"ID":"99d65925-7457-4832-b0e5-8cb75689824f","Type":"ContainerDied","Data":"9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1"}
Sep 15 13:02:12.617658 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.617557    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65f48575b5-qhgwt"
Sep 15 13:02:12.617658 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.617559    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65f48575b5-qhgwt" event={"ID":"99d65925-7457-4832-b0e5-8cb75689824f","Type":"ContainerDied","Data":"e1aac0e2f19f9a3a1f7f86a5d063b13f0c0e41d95a92c631fa6f95a9a587118e"}
Sep 15 13:02:12.628076 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.628054    2586 scope.go:117] "RemoveContainer" containerID="fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718"
Sep 15 13:02:12.636918 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.636900    2586 scope.go:117] "RemoveContainer" containerID="426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f"
Sep 15 13:02:12.637192 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:02:12.637173    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f\": container with ID starting with 426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f not found: ID does not exist" containerID="426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f"
Sep 15 13:02:12.637246 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.637201    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f"} err="failed to get container status \"426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f\": rpc error: code = NotFound desc = could not find container \"426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f\": container with ID starting with 426f5eee9ffefdd00aa42749893f28738b1ef88398b0b6dfb262f906a5d6061f not found: ID does not exist"
Sep 15 13:02:12.637246 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.637217    2586 scope.go:117] "RemoveContainer" containerID="fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718"
Sep 15 13:02:12.637487 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:02:12.637469    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718\": container with ID starting with fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718 not found: ID does not exist" containerID="fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718"
Sep 15 13:02:12.637532 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.637493    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718"} err="failed to get container status \"fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718\": rpc error: code = NotFound desc = could not find container \"fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718\": container with ID starting with fd6cc283ad3bfc06cbe851e61accdd1b02cff1e8435466a4bd28d8a680293718 not found: ID does not exist"
Sep 15 13:02:12.637532 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.637509    2586 scope.go:117] "RemoveContainer" containerID="9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1"
Sep 15 13:02:12.643682 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.643662    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65f48575b5-qhgwt"]
Sep 15 13:02:12.645429 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.645410    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65f48575b5-qhgwt"]
Sep 15 13:02:12.646064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.646050    2586 scope.go:117] "RemoveContainer" containerID="9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1"
Sep 15 13:02:12.646340 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:02:12.646323    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1\": container with ID starting with 9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1 not found: ID does not exist" containerID="9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1"
Sep 15 13:02:12.646408 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.646344    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1"} err="failed to get container status \"9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1\": rpc error: code = NotFound desc = could not find container \"9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1\": container with ID starting with 9b0b08b779c5e0444463978393dfeb5269c1683528881c12e01fd269fbee0fd1 not found: ID does not exist"
Sep 15 13:02:12.652162 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.652143    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d65925-7457-4832-b0e5-8cb75689824f" path="/var/lib/kubelet/pods/99d65925-7457-4832-b0e5-8cb75689824f/volumes"
Sep 15 13:02:12.661111 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.661089    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"]
Sep 15 13:02:12.664188 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:12.664164    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-17f8f8-openshift-default-7f785868d6-ns662"]
Sep 15 13:02:14.652405 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:02:14.652359    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" path="/var/lib/kubelet/pods/17735be7-24d6-49ae-8c6a-96ee853b89e2/volumes"
Sep 15 13:03:13.417292 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417258    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65f48575b5-pkjsv"]
Sep 15 13:03:13.417697 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417629    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6eb26d36-d1b2-4adc-96db-9283925a5643" containerName="payload-pre-processing"
Sep 15 13:03:13.417697 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417640    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6eb26d36-d1b2-4adc-96db-9283925a5643" containerName="payload-pre-processing"
Sep 15 13:03:13.417697 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417654    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cc3bc70-6876-41a0-8307-53507f1e6db9" containerName="payload-processing"
Sep 15 13:03:13.417697 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417659    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cc3bc70-6876-41a0-8307-53507f1e6db9" containerName="payload-processing"
Sep 15 13:03:13.417697 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417667    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c2371c7-2225-4596-b43d-dac23652859b" containerName="payload-pre-processing"
Sep 15 13:03:13.417697 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417673    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c2371c7-2225-4596-b43d-dac23652859b" containerName="payload-pre-processing"
Sep 15 13:03:13.417697 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417678    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy"
Sep 15 13:03:13.417697 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417683    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy"
Sep 15 13:03:13.417697 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417694    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ec550d-4112-4b55-87b0-5318a1ade216" containerName="maas-api"
Sep 15 13:03:13.417697 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417699    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ec550d-4112-4b55-87b0-5318a1ade216" containerName="maas-api"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417705    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ac415e7-2c6c-4679-86bb-e2536f4ef3e6" containerName="payload-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417711    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ac415e7-2c6c-4679-86bb-e2536f4ef3e6" containerName="payload-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417717    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="287fc015-568a-40cf-bb2a-0f4e46470858" containerName="payload-pre-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417722    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="287fc015-568a-40cf-bb2a-0f4e46470858" containerName="payload-pre-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417729    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="751c8728-8c7e-4183-9e53-6a64f084cc1b" containerName="maas-api"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417734    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="751c8728-8c7e-4183-9e53-6a64f084cc1b" containerName="maas-api"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417742    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41954fc4-a5d6-45a1-bd0b-4b507d7e598f" containerName="payload-pre-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417747    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="41954fc4-a5d6-45a1-bd0b-4b507d7e598f" containerName="payload-pre-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417755    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d65925-7457-4832-b0e5-8cb75689824f" containerName="manager"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417760    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d65925-7457-4832-b0e5-8cb75689824f" containerName="manager"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417838    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="751c8728-8c7e-4183-9e53-6a64f084cc1b" containerName="maas-api"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417846    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ac415e7-2c6c-4679-86bb-e2536f4ef3e6" containerName="payload-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417852    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417858    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417863    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="41954fc4-a5d6-45a1-bd0b-4b507d7e598f" containerName="payload-pre-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417868    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cc3bc70-6876-41a0-8307-53507f1e6db9" containerName="payload-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417873    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6eb26d36-d1b2-4adc-96db-9283925a5643" containerName="payload-pre-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417879    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d65925-7457-4832-b0e5-8cb75689824f" containerName="manager"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417885    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="287fc015-568a-40cf-bb2a-0f4e46470858" containerName="payload-pre-processing"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417890    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ec550d-4112-4b55-87b0-5318a1ade216" containerName="maas-api"
Sep 15 13:03:13.418064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.417895    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c2371c7-2225-4596-b43d-dac23652859b" containerName="payload-pre-processing"
Sep 15 13:03:13.420644 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.420629    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:13.422987 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.422968    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 13:03:13.423099 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.422972    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 13:03:13.424000 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.423982    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m2skh\""
Sep 15 13:03:13.424057 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.423982    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 13:03:13.432109 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.432087    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65f48575b5-pkjsv"]
Sep 15 13:03:13.477794 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.477754    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-webhook-cert\") pod \"maas-controller-65f48575b5-pkjsv\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") " pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:13.477911 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.477808    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d86b\" (UniqueName: \"kubernetes.io/projected/0e843296-5695-43ff-bc78-2982a3c1ce6b-kube-api-access-9d86b\") pod \"maas-controller-65f48575b5-pkjsv\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") " pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:13.477911 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.477898    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-metrics-certs\") pod \"maas-controller-65f48575b5-pkjsv\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") " pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:13.578638 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.578610    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-metrics-certs\") pod \"maas-controller-65f48575b5-pkjsv\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") " pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:13.578802 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.578668    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-webhook-cert\") pod \"maas-controller-65f48575b5-pkjsv\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") " pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:13.578802 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.578692    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d86b\" (UniqueName: \"kubernetes.io/projected/0e843296-5695-43ff-bc78-2982a3c1ce6b-kube-api-access-9d86b\") pod \"maas-controller-65f48575b5-pkjsv\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") " pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:13.581083 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.581055    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-webhook-cert\") pod \"maas-controller-65f48575b5-pkjsv\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") " pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:13.581207 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.581186    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-metrics-certs\") pod \"maas-controller-65f48575b5-pkjsv\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") " pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:13.587118 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.587094    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d86b\" (UniqueName: \"kubernetes.io/projected/0e843296-5695-43ff-bc78-2982a3c1ce6b-kube-api-access-9d86b\") pod \"maas-controller-65f48575b5-pkjsv\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") " pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:13.731751 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:13.731666    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:14.004912 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:14.004883    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65f48575b5-pkjsv"]
Sep 15 13:03:14.006207 ip-10-0-141-20 kubenswrapper[2586]: W0915 13:03:14.006179    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e843296_5695_43ff_bc78_2982a3c1ce6b.slice/crio-bd92d1a0b397bf952d3f53c084cd68acd3c7fc1c22309e712c4f18f98fe711be WatchSource:0}: Error finding container bd92d1a0b397bf952d3f53c084cd68acd3c7fc1c22309e712c4f18f98fe711be: Status 404 returned error can't find the container with id bd92d1a0b397bf952d3f53c084cd68acd3c7fc1c22309e712c4f18f98fe711be
Sep 15 13:03:14.007512 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:14.007487    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 13:03:14.906508 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:14.906425    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65f48575b5-pkjsv" event={"ID":"0e843296-5695-43ff-bc78-2982a3c1ce6b","Type":"ContainerStarted","Data":"decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1"}
Sep 15 13:03:14.906508 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:14.906477    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65f48575b5-pkjsv" event={"ID":"0e843296-5695-43ff-bc78-2982a3c1ce6b","Type":"ContainerStarted","Data":"bd92d1a0b397bf952d3f53c084cd68acd3c7fc1c22309e712c4f18f98fe711be"}
Sep 15 13:03:14.906508 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:14.906509    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:14.922416 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:14.922371    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65f48575b5-pkjsv" podStartSLOduration=1.3968874900000001 podStartE2EDuration="1.922354423s" totalImagesPullingTime="525.466933ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:03:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:03:14.921647915 +0000 UTC m=+2262.733559359" watchObservedRunningTime="2026-09-15 13:03:14.922354423 +0000 UTC m=+2262.734265880"
Sep 15 13:03:25.913355 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:25.913323    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:37.404827 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.404760    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65f48575b5-pkjsv"]
Sep 15 13:03:37.405365 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.405053    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65f48575b5-pkjsv" podUID="0e843296-5695-43ff-bc78-2982a3c1ce6b" containerName="manager" containerID="cri-o://decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1" gracePeriod=30
Sep 15 13:03:37.656024 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.655962    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:37.790653 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.790626    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-webhook-cert\") pod \"0e843296-5695-43ff-bc78-2982a3c1ce6b\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") "
Sep 15 13:03:37.790653 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.790661    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-metrics-certs\") pod \"0e843296-5695-43ff-bc78-2982a3c1ce6b\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") "
Sep 15 13:03:37.790925 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.790721    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d86b\" (UniqueName: \"kubernetes.io/projected/0e843296-5695-43ff-bc78-2982a3c1ce6b-kube-api-access-9d86b\") pod \"0e843296-5695-43ff-bc78-2982a3c1ce6b\" (UID: \"0e843296-5695-43ff-bc78-2982a3c1ce6b\") "
Sep 15 13:03:37.792764 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.792735    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e843296-5695-43ff-bc78-2982a3c1ce6b-kube-api-access-9d86b" (OuterVolumeSpecName: "kube-api-access-9d86b") pod "0e843296-5695-43ff-bc78-2982a3c1ce6b" (UID: "0e843296-5695-43ff-bc78-2982a3c1ce6b"). InnerVolumeSpecName "kube-api-access-9d86b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:03:37.792891 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.792802    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0e843296-5695-43ff-bc78-2982a3c1ce6b" (UID: "0e843296-5695-43ff-bc78-2982a3c1ce6b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:03:37.792891 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.792816    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0e843296-5695-43ff-bc78-2982a3c1ce6b" (UID: "0e843296-5695-43ff-bc78-2982a3c1ce6b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:03:37.892099 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.892051    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d86b\" (UniqueName: \"kubernetes.io/projected/0e843296-5695-43ff-bc78-2982a3c1ce6b-kube-api-access-9d86b\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:03:37.892099 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.892093    2586 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-webhook-cert\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:03:37.892099 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:37.892107    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e843296-5695-43ff-bc78-2982a3c1ce6b-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:03:38.024894 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.024862    2586 generic.go:356] "Generic (PLEG): container finished" podID="0e843296-5695-43ff-bc78-2982a3c1ce6b" containerID="decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1" exitCode=0
Sep 15 13:03:38.025054 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.024903    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65f48575b5-pkjsv"
Sep 15 13:03:38.025054 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.024952    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65f48575b5-pkjsv" event={"ID":"0e843296-5695-43ff-bc78-2982a3c1ce6b","Type":"ContainerDied","Data":"decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1"}
Sep 15 13:03:38.025054 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.025009    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65f48575b5-pkjsv" event={"ID":"0e843296-5695-43ff-bc78-2982a3c1ce6b","Type":"ContainerDied","Data":"bd92d1a0b397bf952d3f53c084cd68acd3c7fc1c22309e712c4f18f98fe711be"}
Sep 15 13:03:38.025054 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.025032    2586 scope.go:117] "RemoveContainer" containerID="decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1"
Sep 15 13:03:38.033695 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.033680    2586 scope.go:117] "RemoveContainer" containerID="decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1"
Sep 15 13:03:38.033975 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:03:38.033958    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1\": container with ID starting with decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1 not found: ID does not exist" containerID="decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1"
Sep 15 13:03:38.034067 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.033985    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1"} err="failed to get container status \"decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1\": rpc error: code = NotFound desc = could not find container \"decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1\": container with ID starting with decd2a85e593e9a842d35e2a7389909334b2bbea69287858c70d25c00e961bd1 not found: ID does not exist"
Sep 15 13:03:38.055241 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.049525    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65f48575b5-pkjsv"]
Sep 15 13:03:38.058375 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.058341    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65f48575b5-pkjsv"]
Sep 15 13:03:38.652543 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.652512    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e843296-5695-43ff-bc78-2982a3c1ce6b" path="/var/lib/kubelet/pods/0e843296-5695-43ff-bc78-2982a3c1ce6b/volumes"
Sep 15 13:03:38.859259 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.859229    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65f48575b5-mpbmm"]
Sep 15 13:03:38.859589 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.859578    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy"
Sep 15 13:03:38.859644 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.859592    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="17735be7-24d6-49ae-8c6a-96ee853b89e2" containerName="istio-proxy"
Sep 15 13:03:38.859644 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.859612    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e843296-5695-43ff-bc78-2982a3c1ce6b" containerName="manager"
Sep 15 13:03:38.859644 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.859617    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e843296-5695-43ff-bc78-2982a3c1ce6b" containerName="manager"
Sep 15 13:03:38.859738 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.859667    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e843296-5695-43ff-bc78-2982a3c1ce6b" containerName="manager"
Sep 15 13:03:38.862538 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.862522    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:38.865621 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.865599    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 13:03:38.865621 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.865614    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 13:03:38.865833 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.865649    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 13:03:38.865833 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.865773    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m2skh\""
Sep 15 13:03:38.871023 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:38.871000    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65f48575b5-mpbmm"]
Sep 15 13:03:39.002623 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.002594    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b14f2d34-4f07-4651-99b2-922d0de6bcdd-webhook-cert\") pod \"maas-controller-65f48575b5-mpbmm\" (UID: \"b14f2d34-4f07-4651-99b2-922d0de6bcdd\") " pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:39.002769 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.002643    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b14f2d34-4f07-4651-99b2-922d0de6bcdd-metrics-certs\") pod \"maas-controller-65f48575b5-mpbmm\" (UID: \"b14f2d34-4f07-4651-99b2-922d0de6bcdd\") " pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:39.002769 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.002688    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pvtw\" (UniqueName: \"kubernetes.io/projected/b14f2d34-4f07-4651-99b2-922d0de6bcdd-kube-api-access-4pvtw\") pod \"maas-controller-65f48575b5-mpbmm\" (UID: \"b14f2d34-4f07-4651-99b2-922d0de6bcdd\") " pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:39.103932 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.103903    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b14f2d34-4f07-4651-99b2-922d0de6bcdd-webhook-cert\") pod \"maas-controller-65f48575b5-mpbmm\" (UID: \"b14f2d34-4f07-4651-99b2-922d0de6bcdd\") " pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:39.104087 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.103958    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b14f2d34-4f07-4651-99b2-922d0de6bcdd-metrics-certs\") pod \"maas-controller-65f48575b5-mpbmm\" (UID: \"b14f2d34-4f07-4651-99b2-922d0de6bcdd\") " pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:39.104087 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.103994    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pvtw\" (UniqueName: \"kubernetes.io/projected/b14f2d34-4f07-4651-99b2-922d0de6bcdd-kube-api-access-4pvtw\") pod \"maas-controller-65f48575b5-mpbmm\" (UID: \"b14f2d34-4f07-4651-99b2-922d0de6bcdd\") " pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:39.106578 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.106544    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b14f2d34-4f07-4651-99b2-922d0de6bcdd-metrics-certs\") pod \"maas-controller-65f48575b5-mpbmm\" (UID: \"b14f2d34-4f07-4651-99b2-922d0de6bcdd\") " pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:39.106791 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.106763    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b14f2d34-4f07-4651-99b2-922d0de6bcdd-webhook-cert\") pod \"maas-controller-65f48575b5-mpbmm\" (UID: \"b14f2d34-4f07-4651-99b2-922d0de6bcdd\") " pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:39.111561 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.111534    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pvtw\" (UniqueName: \"kubernetes.io/projected/b14f2d34-4f07-4651-99b2-922d0de6bcdd-kube-api-access-4pvtw\") pod \"maas-controller-65f48575b5-mpbmm\" (UID: \"b14f2d34-4f07-4651-99b2-922d0de6bcdd\") " pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:39.174395 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.174371    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:39.445428 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:39.445403    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65f48575b5-mpbmm"]
Sep 15 13:03:39.447055 ip-10-0-141-20 kubenswrapper[2586]: W0915 13:03:39.447025    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14f2d34_4f07_4651_99b2_922d0de6bcdd.slice/crio-f729408a54d25ffd099438e953b8fd66f327877a0b7db4094e63d4c411588837 WatchSource:0}: Error finding container f729408a54d25ffd099438e953b8fd66f327877a0b7db4094e63d4c411588837: Status 404 returned error can't find the container with id f729408a54d25ffd099438e953b8fd66f327877a0b7db4094e63d4c411588837
Sep 15 13:03:40.037019 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:40.036980    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65f48575b5-mpbmm" event={"ID":"b14f2d34-4f07-4651-99b2-922d0de6bcdd","Type":"ContainerStarted","Data":"b4326563a544521804b7e5bdd1510408bcb6fca12d58d0ac784f8847074a6314"}
Sep 15 13:03:40.037373 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:40.037039    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65f48575b5-mpbmm" event={"ID":"b14f2d34-4f07-4651-99b2-922d0de6bcdd","Type":"ContainerStarted","Data":"f729408a54d25ffd099438e953b8fd66f327877a0b7db4094e63d4c411588837"}
Sep 15 13:03:40.037373 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:40.037074    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:03:40.056401 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:40.056339    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65f48575b5-mpbmm" podStartSLOduration=1.5892528750000001 podStartE2EDuration="2.056320476s" totalImagesPullingTime="467.067601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:03:40.055824513 +0000 UTC m=+2287.867735955" watchObservedRunningTime="2026-09-15 13:03:40.056320476 +0000 UTC m=+2287.868231918"
Sep 15 13:03:51.044604 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:03:51.044572    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65f48575b5-mpbmm"
Sep 15 13:05:02.531317 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:02.531281    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"]
Sep 15 13:05:02.531924 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:02.531520    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx" podUID="b11269c9-c6b9-40b9-b73f-c964ef8ee7e7" containerName="manager" containerID="cri-o://f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67" gracePeriod=10
Sep 15 13:05:02.764032 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:02.763998    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx" podUID="b11269c9-c6b9-40b9-b73f-c964ef8ee7e7" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.31:8081/readyz\": dial tcp 10.134.0.31:8081: connect: connection refused"
Sep 15 13:05:03.298961 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.298938    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 13:05:03.444235 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.444158    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-extensions-socket-volume\") pod \"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7\" (UID: \"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7\") "
Sep 15 13:05:03.444235 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.444209    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5jcg\" (UniqueName: \"kubernetes.io/projected/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-kube-api-access-g5jcg\") pod \"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7\" (UID: \"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7\") "
Sep 15 13:05:03.444583 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.444557    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b11269c9-c6b9-40b9-b73f-c964ef8ee7e7" (UID: "b11269c9-c6b9-40b9-b73f-c964ef8ee7e7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:03.446225 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.446204    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-kube-api-access-g5jcg" (OuterVolumeSpecName: "kube-api-access-g5jcg") pod "b11269c9-c6b9-40b9-b73f-c964ef8ee7e7" (UID: "b11269c9-c6b9-40b9-b73f-c964ef8ee7e7"). InnerVolumeSpecName "kube-api-access-g5jcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:05:03.456788 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.456750    2586 generic.go:356] "Generic (PLEG): container finished" podID="b11269c9-c6b9-40b9-b73f-c964ef8ee7e7" containerID="f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67" exitCode=0
Sep 15 13:05:03.456888 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.456813    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"
Sep 15 13:05:03.456888 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.456813    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx" event={"ID":"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7","Type":"ContainerDied","Data":"f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67"}
Sep 15 13:05:03.456888 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.456878    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx" event={"ID":"b11269c9-c6b9-40b9-b73f-c964ef8ee7e7","Type":"ContainerDied","Data":"4ef56e78023aa4287b954f59030feb307d8217c3c9a033748096f09944a8d5d9"}
Sep 15 13:05:03.457021 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.456909    2586 scope.go:117] "RemoveContainer" containerID="f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67"
Sep 15 13:05:03.470516 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.470499    2586 scope.go:117] "RemoveContainer" containerID="f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67"
Sep 15 13:05:03.470753 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:05:03.470733    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67\": container with ID starting with f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67 not found: ID does not exist" containerID="f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67"
Sep 15 13:05:03.470820 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.470760    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67"} err="failed to get container status \"f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67\": rpc error: code = NotFound desc = could not find container \"f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67\": container with ID starting with f39ccfb4c4cf040f612b5b23321e2f6f4988772286ebb1707646c5e5e4ff4c67 not found: ID does not exist"
Sep 15 13:05:03.481519 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.481498    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"]
Sep 15 13:05:03.484976 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.484956    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qpxcx"]
Sep 15 13:05:03.545442 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.545416    2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-extensions-socket-volume\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:05:03.545442 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:03.545439    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5jcg\" (UniqueName: \"kubernetes.io/projected/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7-kube-api-access-g5jcg\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:05:04.652047 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:05:04.652004    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b11269c9-c6b9-40b9-b73f-c964ef8ee7e7" path="/var/lib/kubelet/pods/b11269c9-c6b9-40b9-b73f-c964ef8ee7e7/volumes"
Sep 15 13:06:03.216534 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.216439    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"]
Sep 15 13:06:03.217077 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.216934    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b11269c9-c6b9-40b9-b73f-c964ef8ee7e7" containerName="manager"
Sep 15 13:06:03.217077 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.216953    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b11269c9-c6b9-40b9-b73f-c964ef8ee7e7" containerName="manager"
Sep 15 13:06:03.217162 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.217078    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b11269c9-c6b9-40b9-b73f-c964ef8ee7e7" containerName="manager"
Sep 15 13:06:03.220188 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.220171    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"
Sep 15 13:06:03.222489 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.222471    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f4mdv\""
Sep 15 13:06:03.235186 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.235156    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"]
Sep 15 13:06:03.346295 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.346236    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfshm\" (UniqueName: \"kubernetes.io/projected/b1fd24c1-d106-4b7b-bb5b-83b518953f3f-kube-api-access-kfshm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-47rkb\" (UID: \"b1fd24c1-d106-4b7b-bb5b-83b518953f3f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"
Sep 15 13:06:03.346502 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.346357    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1fd24c1-d106-4b7b-bb5b-83b518953f3f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-47rkb\" (UID: \"b1fd24c1-d106-4b7b-bb5b-83b518953f3f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"
Sep 15 13:06:03.447373 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.447336    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfshm\" (UniqueName: \"kubernetes.io/projected/b1fd24c1-d106-4b7b-bb5b-83b518953f3f-kube-api-access-kfshm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-47rkb\" (UID: \"b1fd24c1-d106-4b7b-bb5b-83b518953f3f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"
Sep 15 13:06:03.447552 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.447406    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1fd24c1-d106-4b7b-bb5b-83b518953f3f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-47rkb\" (UID: \"b1fd24c1-d106-4b7b-bb5b-83b518953f3f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"
Sep 15 13:06:03.447832 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.447814    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1fd24c1-d106-4b7b-bb5b-83b518953f3f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-47rkb\" (UID: \"b1fd24c1-d106-4b7b-bb5b-83b518953f3f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"
Sep 15 13:06:03.472721 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.472646    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfshm\" (UniqueName: \"kubernetes.io/projected/b1fd24c1-d106-4b7b-bb5b-83b518953f3f-kube-api-access-kfshm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-47rkb\" (UID: \"b1fd24c1-d106-4b7b-bb5b-83b518953f3f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"
Sep 15 13:06:03.529521 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.529489    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"
Sep 15 13:06:03.803843 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:03.803816    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"]
Sep 15 13:06:03.806231 ip-10-0-141-20 kubenswrapper[2586]: W0915 13:06:03.806203    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1fd24c1_d106_4b7b_bb5b_83b518953f3f.slice/crio-90c83b5b64e45a431d124e3e636be53afb1cdc076c971dbf83e3f9b08b488428 WatchSource:0}: Error finding container 90c83b5b64e45a431d124e3e636be53afb1cdc076c971dbf83e3f9b08b488428: Status 404 returned error can't find the container with id 90c83b5b64e45a431d124e3e636be53afb1cdc076c971dbf83e3f9b08b488428
Sep 15 13:06:04.756542 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:04.756499    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb" event={"ID":"b1fd24c1-d106-4b7b-bb5b-83b518953f3f","Type":"ContainerStarted","Data":"d520c472bdc6974377faf527bb61b3132f9f92056891b2855a5c079f699ac5c6"}
Sep 15 13:06:04.757001 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:04.756562    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb" event={"ID":"b1fd24c1-d106-4b7b-bb5b-83b518953f3f","Type":"ContainerStarted","Data":"90c83b5b64e45a431d124e3e636be53afb1cdc076c971dbf83e3f9b08b488428"}
Sep 15 13:06:04.757001 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:04.756644    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"
Sep 15 13:06:04.777755 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:04.777697    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb" podStartSLOduration=1.777680878 podStartE2EDuration="1.777680878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:06:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:06:04.775628176 +0000 UTC m=+2432.587539617" watchObservedRunningTime="2026-09-15 13:06:04.777680878 +0000 UTC m=+2432.589592321"
Sep 15 13:06:15.764678 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:15.764639    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-47rkb"
Sep 15 13:06:51.352156 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.352120    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"]
Sep 15 13:06:51.356215 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.356195    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.358643 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.358618    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-b8c597a1-kserve-self-signed-certs\""
Sep 15 13:06:51.366198 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.366172    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"]
Sep 15 13:06:51.424128 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.424084    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-dshm\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.424328 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.424139    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-model-cache\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.424328 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.424230    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4mpm\" (UniqueName: \"kubernetes.io/projected/3d1265d1-51a3-4dda-a41c-2c0fd744e405-kube-api-access-h4mpm\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.424328 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.424293    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tmp-dir\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.424500 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.424352    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-home\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.424500 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.424391    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tls-certs\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.525351 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.525315    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4mpm\" (UniqueName: \"kubernetes.io/projected/3d1265d1-51a3-4dda-a41c-2c0fd744e405-kube-api-access-h4mpm\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.525558 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.525366    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tmp-dir\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.525558 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.525402    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-home\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.525558 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.525420    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tls-certs\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.525558 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.525546    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-dshm\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.525766 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.525607    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-model-cache\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.525859 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.525840    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-home\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.525911 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.525869    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tmp-dir\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.526010 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.525991    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-model-cache\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.527840 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.527815    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-dshm\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.528097 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.528078    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tls-certs\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.537994 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.537969    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4mpm\" (UniqueName: \"kubernetes.io/projected/3d1265d1-51a3-4dda-a41c-2c0fd744e405-kube-api-access-h4mpm\") pod \"e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") " pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.667049 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.666959    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:06:51.941750 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:51.941723    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"]
Sep 15 13:06:51.943657 ip-10-0-141-20 kubenswrapper[2586]: W0915 13:06:51.943628    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d1265d1_51a3_4dda_a41c_2c0fd744e405.slice/crio-428b02afd7ccf34151f7cdf7876d64bdbedf299975d588c4a6801b5ad94f0825 WatchSource:0}: Error finding container 428b02afd7ccf34151f7cdf7876d64bdbedf299975d588c4a6801b5ad94f0825: Status 404 returned error can't find the container with id 428b02afd7ccf34151f7cdf7876d64bdbedf299975d588c4a6801b5ad94f0825
Sep 15 13:06:52.002645 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:52.002610    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg" event={"ID":"3d1265d1-51a3-4dda-a41c-2c0fd744e405","Type":"ContainerStarted","Data":"730c22383daf3d71a5be5c27aeee2e9dc0894c88d6bb4dbbff586221fe180c9d"}
Sep 15 13:06:52.002798 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:52.002662    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg" event={"ID":"3d1265d1-51a3-4dda-a41c-2c0fd744e405","Type":"ContainerStarted","Data":"428b02afd7ccf34151f7cdf7876d64bdbedf299975d588c4a6801b5ad94f0825"}
Sep 15 13:06:52.019092 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:06:52.019029    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg" podStartSLOduration=1.0190103 podStartE2EDuration="1.0190103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:06:52.017594985 +0000 UTC m=+2479.829506428" watchObservedRunningTime="2026-09-15 13:06:52.0190103 +0000 UTC m=+2479.830921779"
Sep 15 13:07:01.667671 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:01.667618    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:07:01.670142 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:01.667693    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:07:01.680671 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:01.680648    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:07:02.064095 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:02.064060    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:07:36.037652 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:36.037612    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"]
Sep 15 13:07:51.058654 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.058575    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg" podUID="3d1265d1-51a3-4dda-a41c-2c0fd744e405" containerName="main" containerID="cri-o://730c22383daf3d71a5be5c27aeee2e9dc0894c88d6bb4dbbff586221fe180c9d" gracePeriod=45
Sep 15 13:07:51.298512 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.298478    2586 generic.go:356] "Generic (PLEG): container finished" podID="3d1265d1-51a3-4dda-a41c-2c0fd744e405" containerID="730c22383daf3d71a5be5c27aeee2e9dc0894c88d6bb4dbbff586221fe180c9d" exitCode=0
Sep 15 13:07:51.298654 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.298551    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg" event={"ID":"3d1265d1-51a3-4dda-a41c-2c0fd744e405","Type":"ContainerDied","Data":"730c22383daf3d71a5be5c27aeee2e9dc0894c88d6bb4dbbff586221fe180c9d"}
Sep 15 13:07:51.311900 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.311842    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:07:51.392251 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392221    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-dshm\") pod \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") "
Sep 15 13:07:51.392434 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392339    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4mpm\" (UniqueName: \"kubernetes.io/projected/3d1265d1-51a3-4dda-a41c-2c0fd744e405-kube-api-access-h4mpm\") pod \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") "
Sep 15 13:07:51.392434 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392361    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-model-cache\") pod \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") "
Sep 15 13:07:51.392434 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392388    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-home\") pod \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") "
Sep 15 13:07:51.392434 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392417    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tmp-dir\") pod \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") "
Sep 15 13:07:51.392648 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392439    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tls-certs\") pod \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\" (UID: \"3d1265d1-51a3-4dda-a41c-2c0fd744e405\") "
Sep 15 13:07:51.392708 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392649    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-home" (OuterVolumeSpecName: "home") pod "3d1265d1-51a3-4dda-a41c-2c0fd744e405" (UID: "3d1265d1-51a3-4dda-a41c-2c0fd744e405"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:07:51.392708 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392675    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d1265d1-51a3-4dda-a41c-2c0fd744e405" (UID: "3d1265d1-51a3-4dda-a41c-2c0fd744e405"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:07:51.392856 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392693    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d1265d1-51a3-4dda-a41c-2c0fd744e405" (UID: "3d1265d1-51a3-4dda-a41c-2c0fd744e405"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:07:51.392856 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392745    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-model-cache\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:07:51.392856 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.392762    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-home\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:07:51.394546 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.394519    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-dshm" (OuterVolumeSpecName: "dshm") pod "3d1265d1-51a3-4dda-a41c-2c0fd744e405" (UID: "3d1265d1-51a3-4dda-a41c-2c0fd744e405"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:07:51.394688 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.394649    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d1265d1-51a3-4dda-a41c-2c0fd744e405-kube-api-access-h4mpm" (OuterVolumeSpecName: "kube-api-access-h4mpm") pod "3d1265d1-51a3-4dda-a41c-2c0fd744e405" (UID: "3d1265d1-51a3-4dda-a41c-2c0fd744e405"). InnerVolumeSpecName "kube-api-access-h4mpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:07:51.394743 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.394696    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d1265d1-51a3-4dda-a41c-2c0fd744e405" (UID: "3d1265d1-51a3-4dda-a41c-2c0fd744e405"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:07:51.493651 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.493618    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4mpm\" (UniqueName: \"kubernetes.io/projected/3d1265d1-51a3-4dda-a41c-2c0fd744e405-kube-api-access-h4mpm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:07:51.493845 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.493666    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tmp-dir\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:07:51.493845 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.493680    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d1265d1-51a3-4dda-a41c-2c0fd744e405-tls-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:07:51.493845 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:51.493689    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d1265d1-51a3-4dda-a41c-2c0fd744e405-dshm\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:07:52.305842 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:52.305799    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg" event={"ID":"3d1265d1-51a3-4dda-a41c-2c0fd744e405","Type":"ContainerDied","Data":"428b02afd7ccf34151f7cdf7876d64bdbedf299975d588c4a6801b5ad94f0825"}
Sep 15 13:07:52.306273 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:52.305861    2586 scope.go:117] "RemoveContainer" containerID="730c22383daf3d71a5be5c27aeee2e9dc0894c88d6bb4dbbff586221fe180c9d"
Sep 15 13:07:52.306273 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:52.305814    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"
Sep 15 13:07:52.327210 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:52.327184    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"]
Sep 15 13:07:52.332744 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:52.332719    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-b8c597a1-kserve-844d54b6c-m42hg"]
Sep 15 13:07:52.652169 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:52.652082    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d1265d1-51a3-4dda-a41c-2c0fd744e405" path="/var/lib/kubelet/pods/3d1265d1-51a3-4dda-a41c-2c0fd744e405/volumes"
Sep 15 13:07:54.190741 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.190697    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-576799898d-nkttz"]
Sep 15 13:07:54.191257 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.191188    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d1265d1-51a3-4dda-a41c-2c0fd744e405" containerName="main"
Sep 15 13:07:54.191257 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.191208    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d1265d1-51a3-4dda-a41c-2c0fd744e405" containerName="main"
Sep 15 13:07:54.191365 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.191328    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d1265d1-51a3-4dda-a41c-2c0fd744e405" containerName="main"
Sep 15 13:07:54.196072 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.196046    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.206887 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.206857    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-576799898d-nkttz"]
Sep 15 13:07:54.322673 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.322625    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbxbj\" (UniqueName: \"kubernetes.io/projected/2b6c257e-0730-4424-8bfc-ef3e78179c30-kube-api-access-pbxbj\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.322673 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.322680    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b6c257e-0730-4424-8bfc-ef3e78179c30-openshift-service-ca\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.322927 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.322747    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b6c257e-0730-4424-8bfc-ef3e78179c30-metrics-certs\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.322927 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.322837    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b6c257e-0730-4424-8bfc-ef3e78179c30-maas-api-tls\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.423567 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.423530    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbxbj\" (UniqueName: \"kubernetes.io/projected/2b6c257e-0730-4424-8bfc-ef3e78179c30-kube-api-access-pbxbj\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.423567 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.423571    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b6c257e-0730-4424-8bfc-ef3e78179c30-openshift-service-ca\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.423838 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.423622    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b6c257e-0730-4424-8bfc-ef3e78179c30-metrics-certs\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.423838 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.423656    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b6c257e-0730-4424-8bfc-ef3e78179c30-maas-api-tls\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.424388 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.424358    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b6c257e-0730-4424-8bfc-ef3e78179c30-openshift-service-ca\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.426207 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.426188    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b6c257e-0730-4424-8bfc-ef3e78179c30-maas-api-tls\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.426292 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.426258    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b6c257e-0730-4424-8bfc-ef3e78179c30-metrics-certs\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.430626 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.430605    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbxbj\" (UniqueName: \"kubernetes.io/projected/2b6c257e-0730-4424-8bfc-ef3e78179c30-kube-api-access-pbxbj\") pod \"maas-api-576799898d-nkttz\" (UID: \"2b6c257e-0730-4424-8bfc-ef3e78179c30\") " pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.509874 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.509833    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:54.781647 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:54.781583    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-576799898d-nkttz"]
Sep 15 13:07:54.784089 ip-10-0-141-20 kubenswrapper[2586]: W0915 13:07:54.784056    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6c257e_0730_4424_8bfc_ef3e78179c30.slice/crio-9bb91b5541d641d32ed9c654bfdccc28d04cb2d4e327cb864ea9c2629e9c2b67 WatchSource:0}: Error finding container 9bb91b5541d641d32ed9c654bfdccc28d04cb2d4e327cb864ea9c2629e9c2b67: Status 404 returned error can't find the container with id 9bb91b5541d641d32ed9c654bfdccc28d04cb2d4e327cb864ea9c2629e9c2b67
Sep 15 13:07:55.324929 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:55.324868    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz" event={"ID":"2b6c257e-0730-4424-8bfc-ef3e78179c30","Type":"ContainerStarted","Data":"9bb91b5541d641d32ed9c654bfdccc28d04cb2d4e327cb864ea9c2629e9c2b67"}
Sep 15 13:07:56.331823 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:56.331772    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz" event={"ID":"2b6c257e-0730-4424-8bfc-ef3e78179c30","Type":"ContainerStarted","Data":"e011efae3554085f0ce6c832264a50fc808b397cabfa1e1974e77e6ebb0cec57"}
Sep 15 13:07:56.332225 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:56.331866    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:07:56.349421 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:07:56.349370    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz" podStartSLOduration=1.825946461 podStartE2EDuration="2.349354827s" totalImagesPullingTime="523.408366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:07:56.347248922 +0000 UTC m=+2544.159160367" watchObservedRunningTime="2026-09-15 13:07:56.349354827 +0000 UTC m=+2544.161266269"
Sep 15 13:08:02.342869 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:02.342839    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-576799898d-nkttz"
Sep 15 13:08:02.385914 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:02.385876    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"]
Sep 15 13:08:02.386509 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:02.386471    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8" podUID="f5c8cdac-9b87-406f-b77a-cf2a8240c67d" containerName="maas-api" containerID="cri-o://2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275" gracePeriod=30
Sep 15 13:08:03.646955 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.646929    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 13:08:03.717162 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.717071    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcvrj\" (UniqueName: \"kubernetes.io/projected/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-kube-api-access-jcvrj\") pod \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") "
Sep 15 13:08:03.717162 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.717134    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-maas-api-tls\") pod \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") "
Sep 15 13:08:03.717162 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.717164    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-openshift-service-ca\") pod \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") "
Sep 15 13:08:03.717449 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.717211    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-metrics-certs\") pod \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\" (UID: \"f5c8cdac-9b87-406f-b77a-cf2a8240c67d\") "
Sep 15 13:08:03.717586 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.717555    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f5c8cdac-9b87-406f-b77a-cf2a8240c67d" (UID: "f5c8cdac-9b87-406f-b77a-cf2a8240c67d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:08:03.719377 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.719354    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f5c8cdac-9b87-406f-b77a-cf2a8240c67d" (UID: "f5c8cdac-9b87-406f-b77a-cf2a8240c67d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:08:03.719482 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.719409    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f5c8cdac-9b87-406f-b77a-cf2a8240c67d" (UID: "f5c8cdac-9b87-406f-b77a-cf2a8240c67d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:08:03.719482 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.719425    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-kube-api-access-jcvrj" (OuterVolumeSpecName: "kube-api-access-jcvrj") pod "f5c8cdac-9b87-406f-b77a-cf2a8240c67d" (UID: "f5c8cdac-9b87-406f-b77a-cf2a8240c67d"). InnerVolumeSpecName "kube-api-access-jcvrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:08:03.818526 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.818490    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-metrics-certs\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:08:03.818526 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.818518    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcvrj\" (UniqueName: \"kubernetes.io/projected/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-kube-api-access-jcvrj\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:08:03.818526 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.818528    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-maas-api-tls\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:08:03.818812 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:03.818538    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5c8cdac-9b87-406f-b77a-cf2a8240c67d-openshift-service-ca\") on node \"ip-10-0-141-20.ec2.internal\" DevicePath \"\""
Sep 15 13:08:04.375863 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:04.375828    2586 generic.go:356] "Generic (PLEG): container finished" podID="f5c8cdac-9b87-406f-b77a-cf2a8240c67d" containerID="2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275" exitCode=0
Sep 15 13:08:04.376043 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:04.375881    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"
Sep 15 13:08:04.376043 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:04.375899    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8" event={"ID":"f5c8cdac-9b87-406f-b77a-cf2a8240c67d","Type":"ContainerDied","Data":"2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275"}
Sep 15 13:08:04.376043 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:04.375952    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-556658ccdd-q62p8" event={"ID":"f5c8cdac-9b87-406f-b77a-cf2a8240c67d","Type":"ContainerDied","Data":"6ed062d87a699bf8e173443151040d25553d6d1d70561aed228f747951d374cb"}
Sep 15 13:08:04.376043 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:04.375982    2586 scope.go:117] "RemoveContainer" containerID="2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275"
Sep 15 13:08:04.387635 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:04.387612    2586 scope.go:117] "RemoveContainer" containerID="2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275"
Sep 15 13:08:04.387937 ip-10-0-141-20 kubenswrapper[2586]: E0915 13:08:04.387913    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275\": container with ID starting with 2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275 not found: ID does not exist" containerID="2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275"
Sep 15 13:08:04.388031 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:04.387945    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275"} err="failed to get container status \"2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275\": rpc error: code = NotFound desc = could not find container \"2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275\": container with ID starting with 2438b7377aa506489e555c29901d86367000f8983b843fbca2eefedb6b401275 not found: ID does not exist"
Sep 15 13:08:04.400027 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:04.400002    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"]
Sep 15 13:08:04.403548 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:04.403522    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-556658ccdd-q62p8"]
Sep 15 13:08:04.652485 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:04.652402    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5c8cdac-9b87-406f-b77a-cf2a8240c67d" path="/var/lib/kubelet/pods/f5c8cdac-9b87-406f-b77a-cf2a8240c67d/volumes"
Sep 15 13:08:30.611900 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:30.611867    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-784kg_c060ad4e-12da-469f-8661-ef92971fe8ad/manager/0.log"
Sep 15 13:08:30.746328 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:30.746292    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vg9wm_1c095478-39f7-402e-8be0-37aac021dda2/manager/0.log"
Sep 15 13:08:30.861633 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:30.861597    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65f48575b5-mpbmm_b14f2d34-4f07-4651-99b2-922d0de6bcdd/manager/0.log"
Sep 15 13:08:31.220632 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:31.220605    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665cd58c4f-7dhb5_2257ca24-da90-4229-a35c-bac604a3d185/copy-manifests/0.log"
Sep 15 13:08:31.226710 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:31.226686    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665cd58c4f-7dhb5_2257ca24-da90-4229-a35c-bac604a3d185/manager/0.log"
Sep 15 13:08:31.813683 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:31.813649    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-576799898d-nkttz_2b6c257e-0730-4424-8bfc-ef3e78179c30/maas-api/0.log"
Sep 15 13:08:32.262974 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:32.262935    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-f992w_2fcce475-67e1-43b9-90d0-356c2d97e8dd/postgres/0.log"
Sep 15 13:08:33.144449 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:33.144423    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-s7dhm_0840121f-8970-4437-82e1-c0eb8ec392b8/manager/0.log"
Sep 15 13:08:33.252325 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:33.252293    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-snq5m_ad50bc5f-fac3-4fd3-ac1d-4a8455b42f3c/manager/0.log"
Sep 15 13:08:33.476828 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:33.476742    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-47rkb_b1fd24c1-d106-4b7b-bb5b-83b518953f3f/manager/0.log"
Sep 15 13:08:33.746386 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:33.746355    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-jw4bh_e29364af-ff4f-4457-9dc0-dec168150c64/manager/0.log"
Sep 15 13:08:34.440766 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:34.440725    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6956d8dd68-jj2wn_ecdbe354-d2dd-4631-be5c-c08956500a7f/kube-auth-proxy/0.log"
Sep 15 13:08:34.558930 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:34.558899    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-dtgvl_d26fc663-72b4-4ff5-9e56-3d5eb6603e29/istio-proxy/1.log"
Sep 15 13:08:34.673062 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:34.673025    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k7cfc_1a21963b-0a26-4775-83a1-ccbcfbc6a38b/payload-pre-processing/0.log"
Sep 15 13:08:35.387408 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:35.387371    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qm4x6_d304d1ba-1999-4766-8df9-b0ad2d9827e9/main/0.log"
Sep 15 13:08:35.850238 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:35.850206    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-hm4t7_f440992c-7441-4099-8b1f-1a9f07527ce5/main/0.log"
Sep 15 13:08:41.976853 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.976818    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fpbvp/must-gather-bp9dg"]
Sep 15 13:08:41.977348 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.977229    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5c8cdac-9b87-406f-b77a-cf2a8240c67d" containerName="maas-api"
Sep 15 13:08:41.977348 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.977242    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5c8cdac-9b87-406f-b77a-cf2a8240c67d" containerName="maas-api"
Sep 15 13:08:41.977348 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.977299    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5c8cdac-9b87-406f-b77a-cf2a8240c67d" containerName="maas-api"
Sep 15 13:08:41.980419 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.980403    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fpbvp/must-gather-bp9dg"
Sep 15 13:08:41.982925 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.982898    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fpbvp\"/\"kube-root-ca.crt\""
Sep 15 13:08:41.983064 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.982898    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fpbvp\"/\"openshift-service-ca.crt\""
Sep 15 13:08:41.983611 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.983590    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fpbvp\"/\"default-dockercfg-kswxq\""
Sep 15 13:08:41.990389 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.990367    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fpbvp/must-gather-bp9dg"]
Sep 15 13:08:41.991904 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.991887    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vstx\" (UniqueName: \"kubernetes.io/projected/968ab01f-4ba7-4847-946d-aea9c6da5ea1-kube-api-access-7vstx\") pod \"must-gather-bp9dg\" (UID: \"968ab01f-4ba7-4847-946d-aea9c6da5ea1\") " pod="openshift-must-gather-fpbvp/must-gather-bp9dg"
Sep 15 13:08:41.991972 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:41.991949    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/968ab01f-4ba7-4847-946d-aea9c6da5ea1-must-gather-output\") pod \"must-gather-bp9dg\" (UID: \"968ab01f-4ba7-4847-946d-aea9c6da5ea1\") " pod="openshift-must-gather-fpbvp/must-gather-bp9dg"
Sep 15 13:08:42.093226 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:42.093191    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/968ab01f-4ba7-4847-946d-aea9c6da5ea1-must-gather-output\") pod \"must-gather-bp9dg\" (UID: \"968ab01f-4ba7-4847-946d-aea9c6da5ea1\") " pod="openshift-must-gather-fpbvp/must-gather-bp9dg"
Sep 15 13:08:42.093395 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:42.093356    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vstx\" (UniqueName: \"kubernetes.io/projected/968ab01f-4ba7-4847-946d-aea9c6da5ea1-kube-api-access-7vstx\") pod \"must-gather-bp9dg\" (UID: \"968ab01f-4ba7-4847-946d-aea9c6da5ea1\") " pod="openshift-must-gather-fpbvp/must-gather-bp9dg"
Sep 15 13:08:42.093555 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:42.093536    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/968ab01f-4ba7-4847-946d-aea9c6da5ea1-must-gather-output\") pod \"must-gather-bp9dg\" (UID: \"968ab01f-4ba7-4847-946d-aea9c6da5ea1\") " pod="openshift-must-gather-fpbvp/must-gather-bp9dg"
Sep 15 13:08:42.106451 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:42.106422    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vstx\" (UniqueName: \"kubernetes.io/projected/968ab01f-4ba7-4847-946d-aea9c6da5ea1-kube-api-access-7vstx\") pod \"must-gather-bp9dg\" (UID: \"968ab01f-4ba7-4847-946d-aea9c6da5ea1\") " pod="openshift-must-gather-fpbvp/must-gather-bp9dg"
Sep 15 13:08:42.290462 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:42.290428    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fpbvp/must-gather-bp9dg"
Sep 15 13:08:42.574542 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:42.574519    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fpbvp/must-gather-bp9dg"]
Sep 15 13:08:42.576420 ip-10-0-141-20 kubenswrapper[2586]: W0915 13:08:42.576389    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod968ab01f_4ba7_4847_946d_aea9c6da5ea1.slice/crio-d62e01fe16efc7aaeaecd14474cbbabe065b49557a3686da8c039d54219fb339 WatchSource:0}: Error finding container d62e01fe16efc7aaeaecd14474cbbabe065b49557a3686da8c039d54219fb339: Status 404 returned error can't find the container with id d62e01fe16efc7aaeaecd14474cbbabe065b49557a3686da8c039d54219fb339
Sep 15 13:08:42.578635 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:42.578617    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 13:08:43.567814 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:43.567752    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fpbvp/must-gather-bp9dg" event={"ID":"968ab01f-4ba7-4847-946d-aea9c6da5ea1","Type":"ContainerStarted","Data":"39c0a138762a211e7bdaae8a7ffa3aa4c6cde8af635fd97e7a9be297d8442b08"}
Sep 15 13:08:43.568253 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:43.567827    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fpbvp/must-gather-bp9dg" event={"ID":"968ab01f-4ba7-4847-946d-aea9c6da5ea1","Type":"ContainerStarted","Data":"9a988e32e7c4767fc80044444e1331bb3874789f174bbbe00d0c7c7ff186e4b6"}
Sep 15 13:08:43.568253 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:43.567848    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fpbvp/must-gather-bp9dg" event={"ID":"968ab01f-4ba7-4847-946d-aea9c6da5ea1","Type":"ContainerStarted","Data":"d62e01fe16efc7aaeaecd14474cbbabe065b49557a3686da8c039d54219fb339"}
Sep 15 13:08:43.583861 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:43.583808    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fpbvp/must-gather-bp9dg" podStartSLOduration=1.8212453499999999 podStartE2EDuration="2.583766916s" totalImagesPullingTime="762.521566ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:08:43.58225694 +0000 UTC m=+2591.394168393" watchObservedRunningTime="2026-09-15 13:08:43.583766916 +0000 UTC m=+2591.395678416"
Sep 15 13:08:44.853190 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:44.853142    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2n7pd_ca49e661-dd33-4c7c-9751-408033360944/global-pull-secret-syncer/0.log"
Sep 15 13:08:45.005937 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:45.005887    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hcnk6_94cf40a2-35af-402c-b86d-f023b2ba9c85/konnectivity-agent/0.log"
Sep 15 13:08:45.110463 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:45.110352    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-20.ec2.internal_4baa257649766985809baa75fc95af2e/haproxy/0.log"
Sep 15 13:08:50.248928 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:50.248825    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-s7dhm_0840121f-8970-4437-82e1-c0eb8ec392b8/manager/0.log"
Sep 15 13:08:50.274027 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:50.273966    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-snq5m_ad50bc5f-fac3-4fd3-ac1d-4a8455b42f3c/manager/0.log"
Sep 15 13:08:50.346703 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:50.346672    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-47rkb_b1fd24c1-d106-4b7b-bb5b-83b518953f3f/manager/0.log"
Sep 15 13:08:50.451701 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:50.451663    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-jw4bh_e29364af-ff4f-4457-9dc0-dec168150c64/manager/0.log"
Sep 15 13:08:52.173169 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.173134    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-qj647_788827a0-a45b-487e-9459-c52473975518/kube-state-metrics/0.log"
Sep 15 13:08:52.190995 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.190933    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-qj647_788827a0-a45b-487e-9459-c52473975518/kube-rbac-proxy-main/0.log"
Sep 15 13:08:52.213922 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.213858    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-qj647_788827a0-a45b-487e-9459-c52473975518/kube-rbac-proxy-self/0.log"
Sep 15 13:08:52.241176 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.241122    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-766f4cc575-zr9wp_9cd3f9dc-3d5a-46cb-966c-ec93090cd379/metrics-server/0.log"
Sep 15 13:08:52.492518 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.492456    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nptnb_e00a1b6f-a485-437c-938d-70075b2fb20f/node-exporter/0.log"
Sep 15 13:08:52.522272 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.522224    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nptnb_e00a1b6f-a485-437c-938d-70075b2fb20f/kube-rbac-proxy/0.log"
Sep 15 13:08:52.547233 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.547200    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nptnb_e00a1b6f-a485-437c-938d-70075b2fb20f/init-textfile/0.log"
Sep 15 13:08:52.677385 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.677299    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28ee608a-2903-496a-acf4-de4a9289ab34/prometheus/0.log"
Sep 15 13:08:52.699814 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.699760    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28ee608a-2903-496a-acf4-de4a9289ab34/config-reloader/0.log"
Sep 15 13:08:52.720733 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.720701    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28ee608a-2903-496a-acf4-de4a9289ab34/thanos-sidecar/0.log"
Sep 15 13:08:52.745739 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.745664    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28ee608a-2903-496a-acf4-de4a9289ab34/kube-rbac-proxy-web/0.log"
Sep 15 13:08:52.766281 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.766218    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28ee608a-2903-496a-acf4-de4a9289ab34/kube-rbac-proxy/0.log"
Sep 15 13:08:52.787363 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.787298    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28ee608a-2903-496a-acf4-de4a9289ab34/kube-rbac-proxy-thanos/0.log"
Sep 15 13:08:52.809569 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.809522    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28ee608a-2903-496a-acf4-de4a9289ab34/init-config-reloader/0.log"
Sep 15 13:08:52.893330 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:52.893287    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-2zchc_78bbf197-6b79-4b69-966a-4a81a9c5946b/prometheus-operator-admission-webhook/0.log"
Sep 15 13:08:55.457700 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.455767    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"]
Sep 15 13:08:55.461810 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.461764    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.468603 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.468572    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"]
Sep 15 13:08:55.546700 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.546649    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-proc\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.546919 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.546713    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-lib-modules\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.546919 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.546848    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-podres\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.546919 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.546912    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb7vh\" (UniqueName: \"kubernetes.io/projected/30a25eee-b846-4140-8b23-ac04ebd4320b-kube-api-access-cb7vh\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.547094 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.547010    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-sys\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.624731 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.624696    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-c5f54574c-v9kl9_bb7a6636-e72e-4a8e-aadc-f3d253df10e4/console/0.log"
Sep 15 13:08:55.648342 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.648294    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-proc\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.648539 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.648362    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-lib-modules\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.648539 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.648432    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-podres\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.648539 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.648511    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb7vh\" (UniqueName: \"kubernetes.io/projected/30a25eee-b846-4140-8b23-ac04ebd4320b-kube-api-access-cb7vh\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.648721 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.648624    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-sys\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.648789 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.648738    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-sys\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.648852 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.648814    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-proc\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.648963 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.648941    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-lib-modules\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.649089 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.649067    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30a25eee-b846-4140-8b23-ac04ebd4320b-podres\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.657892 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.657861    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb7vh\" (UniqueName: \"kubernetes.io/projected/30a25eee-b846-4140-8b23-ac04ebd4320b-kube-api-access-cb7vh\") pod \"perf-node-gather-daemonset-t59l6\" (UID: \"30a25eee-b846-4140-8b23-ac04ebd4320b\") " pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:55.776582 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:55.776545    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:56.066924 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:56.066806    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"]
Sep 15 13:08:56.070115 ip-10-0-141-20 kubenswrapper[2586]: W0915 13:08:56.070077    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod30a25eee_b846_4140_8b23_ac04ebd4320b.slice/crio-19610317a0ce650bf19fee58a0a2c93f7497e020bd3a7389f7b48b7d72cbd8a8 WatchSource:0}: Error finding container 19610317a0ce650bf19fee58a0a2c93f7497e020bd3a7389f7b48b7d72cbd8a8: Status 404 returned error can't find the container with id 19610317a0ce650bf19fee58a0a2c93f7497e020bd3a7389f7b48b7d72cbd8a8
Sep 15 13:08:56.657328 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:56.657281    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6" event={"ID":"30a25eee-b846-4140-8b23-ac04ebd4320b","Type":"ContainerStarted","Data":"3fe01899175d6d45c3c2dcee08395ae62d51670bd73337bada0c3c400f2ef068"}
Sep 15 13:08:56.657840 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:56.657344    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6" event={"ID":"30a25eee-b846-4140-8b23-ac04ebd4320b","Type":"ContainerStarted","Data":"19610317a0ce650bf19fee58a0a2c93f7497e020bd3a7389f7b48b7d72cbd8a8"}
Sep 15 13:08:56.657840 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:56.657469    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:08:56.674092 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:56.674025    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6" podStartSLOduration=1.674004649 podStartE2EDuration="1.674004649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 13:08:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:08:56.672553874 +0000 UTC m=+2604.484465318" watchObservedRunningTime="2026-09-15 13:08:56.674004649 +0000 UTC m=+2604.485916092"
Sep 15 13:08:57.501347 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:57.501315    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v8q24_d76f9797-13c3-49fa-b018-65de900b8bb3/dns/0.log"
Sep 15 13:08:57.521554 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:57.521526    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v8q24_d76f9797-13c3-49fa-b018-65de900b8bb3/kube-rbac-proxy/0.log"
Sep 15 13:08:57.588137 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:57.588106    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mqn5j_6b7ec0a1-a68d-4699-bb5d-c3152791a0ef/dns-node-resolver/0.log"
Sep 15 13:08:58.165808 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:08:58.165750    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k28bb_2f5cb386-9dde-4e3c-9122-eb1c6042e0e4/node-ca/0.log"
Sep 15 13:09:00.008719 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:00.008690    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6956d8dd68-jj2wn_ecdbe354-d2dd-4631-be5c-c08956500a7f/kube-auth-proxy/0.log"
Sep 15 13:09:00.034322 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:00.034288    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-dtgvl_d26fc663-72b4-4ff5-9e56-3d5eb6603e29/istio-proxy/0.log"
Sep 15 13:09:00.038029 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:00.038000    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-dtgvl_d26fc663-72b4-4ff5-9e56-3d5eb6603e29/istio-proxy/1.log"
Sep 15 13:09:00.070231 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:00.070197    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k7cfc_1a21963b-0a26-4775-83a1-ccbcfbc6a38b/payload-pre-processing/0.log"
Sep 15 13:09:01.122358 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:01.122331    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tp5lb_040b0ab2-3bbc-4355-acf8-b78f7446bdf2/serve-healthcheck-canary/0.log"
Sep 15 13:09:01.598696 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:01.598665    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5jrd8_825edbd7-47cb-4d16-ac7a-34db724a8f13/kube-rbac-proxy/0.log"
Sep 15 13:09:01.620742 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:01.620713    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5jrd8_825edbd7-47cb-4d16-ac7a-34db724a8f13/exporter/0.log"
Sep 15 13:09:01.647278 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:01.647252    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5jrd8_825edbd7-47cb-4d16-ac7a-34db724a8f13/extractor/0.log"
Sep 15 13:09:02.676123 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:02.676095    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fpbvp/perf-node-gather-daemonset-t59l6"
Sep 15 13:09:03.864849 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:03.864813    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-784kg_c060ad4e-12da-469f-8661-ef92971fe8ad/manager/0.log"
Sep 15 13:09:04.246293 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:04.246263    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vg9wm_1c095478-39f7-402e-8be0-37aac021dda2/manager/0.log"
Sep 15 13:09:04.308283 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:04.308251    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65f48575b5-mpbmm_b14f2d34-4f07-4651-99b2-922d0de6bcdd/manager/0.log"
Sep 15 13:09:04.500046 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:04.499960    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665cd58c4f-7dhb5_2257ca24-da90-4229-a35c-bac604a3d185/manager/0.log"
Sep 15 13:09:04.526013 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:04.525984    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665cd58c4f-7dhb5_2257ca24-da90-4229-a35c-bac604a3d185/copy-manifests/0.log"
Sep 15 13:09:06.838764 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:06.838696    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6956d8dd68-jj2wn_ecdbe354-d2dd-4631-be5c-c08956500a7f/kube-auth-proxy/0.log"
Sep 15 13:09:06.861166 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:06.860548    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-dtgvl_d26fc663-72b4-4ff5-9e56-3d5eb6603e29/istio-proxy/0.log"
Sep 15 13:09:06.865310 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:06.865253    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-dtgvl_d26fc663-72b4-4ff5-9e56-3d5eb6603e29/istio-proxy/1.log"
Sep 15 13:09:06.886921 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:06.886872    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k7cfc_1a21963b-0a26-4775-83a1-ccbcfbc6a38b/payload-pre-processing/0.log"
Sep 15 13:09:09.266693 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:09.266667    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6956d8dd68-jj2wn_ecdbe354-d2dd-4631-be5c-c08956500a7f/kube-auth-proxy/0.log"
Sep 15 13:09:09.292901 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:09.292868    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-dtgvl_d26fc663-72b4-4ff5-9e56-3d5eb6603e29/istio-proxy/0.log"
Sep 15 13:09:09.297518 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:09.297477    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-dtgvl_d26fc663-72b4-4ff5-9e56-3d5eb6603e29/istio-proxy/1.log"
Sep 15 13:09:09.322826 ip-10-0-141-20 kubenswrapper[2586]: I0915 13:09:09.322801    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k7cfc_1a21963b-0a26-4775-83a1-ccbcfbc6a38b/payload-pre-processing/0.log"