Sep 09 20:32:18.646152 ip-10-0-140-126 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 20:32:19.031557 ip-10-0-140-126 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:32:19.031557 ip-10-0-140-126 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 20:32:19.031557 ip-10-0-140-126 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:32:19.031557 ip-10-0-140-126 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 20:32:19.031557 ip-10-0-140-126 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:32:19.033426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.033321    2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 20:32:19.036252 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036237    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:32:19.036252 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036251    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036255    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036259    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036262    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036266    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036269    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036272    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036275    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036277    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036281    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036286    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036290    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036293    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036296    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036300    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036303    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036306    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036310    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036312    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:32:19.036316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036315    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036318    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036320    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036323    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036327    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036329    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036332    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036335    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036338    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036340    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036343    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036346    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036348    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036351    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036353    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036356    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036358    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036361    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036363    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036366    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:32:19.036755 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036369    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036372    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036374    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036377    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036379    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036382    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036384    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036387    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036391    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036395    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036397    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036400    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036402    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036406    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036409    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036412    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036415    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036417    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036420    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036423    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:32:19.037273 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036425    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036428    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036431    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036433    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036436    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036438    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036441    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036443    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036446    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036448    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036451    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036454    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036456    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036459    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036461    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036464    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036466    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036469    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036471    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036474    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:32:19.037884 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036476    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036479    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036481    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036485    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036488    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036491    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036494    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036497    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036501    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036503    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036506    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036509    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036511    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036514    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036516    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036519    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036521    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036524    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.036526    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037238    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:32:19.038361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037245    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037248    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037251    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037254    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037262    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037265    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037267    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037270    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037272    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037276    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037281    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037284    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037286    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037291    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037295    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037298    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037301    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037304    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037307    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:32:19.038845 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037310    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037313    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037316    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037318    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037321    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037324    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037326    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037329    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037331    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037334    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037336    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037339    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037341    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037344    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037346    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037348    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037351    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037353    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037361    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037365    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:32:19.039327 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037367    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037370    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037373    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037375    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037379    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037382    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037385    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037388    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037391    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037393    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037397    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037400    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037403    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037406    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037408    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037412    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037414    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037417    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037419    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037422    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:32:19.039816 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037424    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037427    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037429    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037432    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037434    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037436    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037439    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037441    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037444    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037446    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037449    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037452    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037454    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037456    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037459    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037462    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037464    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037467    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037469    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037471    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:32:19.040317 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037474    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037476    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037479    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037482    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037485    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037487    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037490    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037492    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037495    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037498    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037501    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037503    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037506    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037508    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037511    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037513    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037516    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037518    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037521    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037942    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:32:19.040800 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037948    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037951    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037954    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037957    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037960    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037963    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037966    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037968    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037971    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037973    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037976    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037978    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037983    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037986    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037989    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037991    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037994    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037996    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.037999    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038003    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:32:19.041316 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038006    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038009    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038013    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038016    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038019    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038022    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038025    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038027    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038030    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038033    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038036    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038039    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038042    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038044    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038047    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038050    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038053    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038055    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038058    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:32:19.041807 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038060    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038063    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038065    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038067    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038070    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038074    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038076    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038079    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038082    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038084    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038086    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038089    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038091    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038094    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038096    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038099    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038103    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038106    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038109    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038112    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:32:19.042272 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038115    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038117    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038120    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038123    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038125    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038128    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038131    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038133    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038136    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038138    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038141    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038144    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038148    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038150    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038153    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038155    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038158    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038163    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038166    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:32:19.042781 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038168    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038171    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038173    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038176    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038178    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038181    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038183    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038185    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038188    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038191    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038193    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038196    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038198    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038200    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038203    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038205    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038208    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038210    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038213    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038215    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:32:19.043250 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038283    2550 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038318    2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038328    2550 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038332    2550 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038337    2550 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038340    2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038344    2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038349    2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038352    2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038355    2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038360    2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038363    2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038366    2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038369    2550 flags.go:64] FLAG: --cgroup-root=""
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038372    2550 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038375    2550 flags.go:64] FLAG: --client-ca-file=""
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038378    2550 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038381    2550 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038385    2550 flags.go:64] FLAG: --cluster-domain=""
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038388    2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038393    2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038396    2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038399    2550 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038403    2550 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 20:32:19.043732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038406    2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038409    2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038412    2550 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038415    2550 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038418    2550 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038420    2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038423    2550 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038426    2550 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038430    2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038433    2550 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038436    2550 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038438    2550 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038442    2550 flags.go:64] FLAG: --enable-server="true"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038445    2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038450    2550 flags.go:64] FLAG: --event-burst="100"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038453    2550 flags.go:64] FLAG: --event-qps="50"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038456    2550 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038459    2550 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038463    2550 flags.go:64] FLAG: --eviction-hard=""
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038466    2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038469    2550 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038472    2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038475    2550 flags.go:64] FLAG: --eviction-soft=""
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038478    2550 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038480    2550 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 20:32:19.044322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038483    2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038486    2550 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038489    2550 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038493    2550 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038496    2550 flags.go:64] FLAG: --feature-gates=""
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038500    2550 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038503    2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038506    2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038509    2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038512    2550 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038515    2550 flags.go:64] FLAG: --help="false"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038518    2550 flags.go:64] FLAG: --hostname-override="ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038521    2550 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038524    2550 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038527    2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038530    2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038534    2550 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038537    2550 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038540    2550 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038542    2550 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038545    2550 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038553    2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038556    2550 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038559    2550 flags.go:64] FLAG: --kube-reserved=""
Sep 09 20:32:19.045025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038562    2550 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038565    2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038568    2550 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038571    2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038574    2550 flags.go:64] FLAG: --lock-file=""
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038576    2550 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038579    2550 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038583    2550 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038592    2550 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038595    2550 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038598    2550 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038602    2550 flags.go:64] FLAG: --logging-format="text"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038605    2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038608    2550 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038611    2550 flags.go:64] FLAG: --manifest-url=""
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038614    2550 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038618    2550 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038621    2550 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038625    2550 flags.go:64] FLAG: --max-pods="110"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038628    2550 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038631    2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038634    2550 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038637    2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038640    2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038643    2550 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 20:32:19.045686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038646    2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038653    2550 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038656    2550 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038659    2550 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038662    2550 flags.go:64] FLAG: --pod-cidr=""
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038666    2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038676    2550 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038679    2550 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038682    2550 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038685    2550 flags.go:64] FLAG: --port="10250"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038688    2550 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038691    2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d31ab5ea733fce40"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038695    2550 flags.go:64] FLAG: --qos-reserved=""
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038697    2550 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038700    2550 flags.go:64] FLAG: --register-node="true"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038703    2550 flags.go:64] FLAG: --register-with-taints=""
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038710    2550 flags.go:64] FLAG: --registry-burst="10"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038713    2550 flags.go:64] FLAG: --registry-qps="5"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038717    2550 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038720    2550 flags.go:64] FLAG: --reserved-memory=""
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038724    2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038727    2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038730    2550 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038732    2550 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038735    2550 flags.go:64] FLAG: --runonce="false"
Sep 09 20:32:19.046336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038738    2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038742    2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038746    2550 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038750    2550 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038754    2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038758    2550 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038761    2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038764    2550 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038766    2550 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038769    2550 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038772    2550 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038775    2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038778    2550 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038781    2550 flags.go:64] FLAG: --system-cgroups=""
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038784    2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038789    2550 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038792    2550 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038795    2550 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038799    2550 flags.go:64] FLAG: --tls-min-version=""
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038801    2550 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038804    2550 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038806    2550 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038810    2550 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038815    2550 flags.go:64] FLAG: --v="2"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038819    2550 flags.go:64] FLAG: --version="false"
Sep 09 20:32:19.046950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038825    2550 flags.go:64] FLAG: --vmodule=""
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038828    2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.038832    2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038960    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038964    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038967    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038970    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038973    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038976    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038979    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038982    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038985    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038988    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038990    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038993    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038995    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.038998    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039001    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039003    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039006    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:32:19.047558 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039008    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039011    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039014    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039017    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039019    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039022    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039025    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039027    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039030    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039033    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039035    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039038    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039042    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039046    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039050    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039052    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039055    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039058    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039060    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:32:19.048054 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039063    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039065    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039069    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039073    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039076    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039079    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039082    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039085    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039088    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039091    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039094    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039097    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039099    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039102    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039105    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039108    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039111    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039113    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039116    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:32:19.048525 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039118    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039121    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039123    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039126    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039128    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039131    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039134    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039138    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039141    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039144    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039147    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039150    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039153    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039156    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039159    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039162    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039165    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039169    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039172    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:32:19.049116 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039174    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039177    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039179    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039182    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039184    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039187    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039189    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039192    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039195    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039197    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039200    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039202    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039205    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039207    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039210    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039213    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039215    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039217    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039220    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039224    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:32:19.049578 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039227    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039229    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039231    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039234    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.039236    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.039242    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.046786    2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.046803    2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.047672    2550 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.047679    2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047743    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047748    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047752    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047755    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047757    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:32:19.050118 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047760    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047763    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047766    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047768    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047771    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047773    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047777    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047781    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047785    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047789    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047793    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047796    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047798    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047801    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047804    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047806    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047809    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047811    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047814    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:32:19.050495 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047817    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047819    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047822    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047824    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047827    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047836    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047838    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047841    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047843    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047846    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047848    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047851    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047853    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047856    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047859    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047861    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047863    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047876    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047879    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047881    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:32:19.050970 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047884    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047887    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047889    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047892    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047895    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047897    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047900    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047903    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047906    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047908    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047911    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047913    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047916    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047918    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047921    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047924    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047926    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047929    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047937    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047940    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:32:19.051458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047942    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047945    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047947    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047950    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047953    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047956    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047958    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047961    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047963    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047966    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047968    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047970    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047973    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047975    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047978    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047980    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047983    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047985    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047988    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:32:19.051951 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047991    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047993    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047996    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.047999    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048001    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048003    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048006    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048009    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048011    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048015    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048019    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048022    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048030    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048033    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048036    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:32:19.052405 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048038    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.048043    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048189    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048194    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048197    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048199    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048203    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048205    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048208    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048210    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048213    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048217    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048220    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048223    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048231    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:32:19.052775 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048234    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048237    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048239    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048242    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048245    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048248    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048250    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048255    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048258    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048261    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048264    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048267    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048269    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048272    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048274    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048285    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048288    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048291    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048293    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:32:19.053167 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048295    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048298    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048301    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048305    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048307    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048310    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048312    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048315    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048317    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048320    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048322    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048325    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048328    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048330    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048333    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048335    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048338    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048341    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048343    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:32:19.053619 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048346    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048348    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048351    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048353    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048356    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048358    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048361    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048363    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048366    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048368    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048378    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048380    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048383    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048385    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048387    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048390    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048392    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048395    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048397    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048400    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:32:19.054157 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048402    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048405    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048407    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048409    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048412    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048414    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048417    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048419    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048422    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048425    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048427    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048430    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048432    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048434    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048437    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048439    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048442    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048445    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048447    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048450    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:32:19.054646 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048452    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048454    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048457    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048465    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048467    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048470    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048472    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:19.048474    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.048479    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.048597    2550 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.050281    2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.051250    2550 server.go:1019] "Starting client certificate rotation"
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.051379    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 20:32:19.055173 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.051412    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 20:32:19.055616 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.054346    2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 20:32:19.062285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.062260    2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 20:32:19.073865 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.073847    2550 log.go:25] "Validated CRI v1 runtime API"
Sep 09 20:32:19.076942 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.076929    2550 log.go:25] "Validated CRI v1 image API"
Sep 09 20:32:19.077756 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.077743    2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 20:32:19.082466 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.082444    2550 fs.go:135] Filesystem UUIDs: map[2c8e2a3e-0f6f-4538-8563-7202b280d324:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 de99d540-70e0-4ece-9dcb-667408294020:/dev/nvme0n1p4]
Sep 09 20:32:19.082538 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.082464    2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 20:32:19.082846 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.082826    2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 20:32:19.087886 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.087771    2550 manager.go:216] Machine: {Timestamp:2026-09-09 20:32:19.086703065 +0000 UTC m=+0.322670140 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109086 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2aafbcab23f24a6db1a06e2a489b0b SystemUUID:ec2aafbc-ab23-f24a-6db1-a06e2a489b0b BootID:61330c49-c3ee-4e43-80c5-7b0edb1b2440 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:86:8a:30:04:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:86:8a:30:04:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:84:03:71:e9:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 20:32:19.087886 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.087886    2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 20:32:19.088043 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.087966    2550 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 20:32:19.091085 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.091060    2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 20:32:19.091227 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.091089    2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-126.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 20:32:19.092194 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.092183    2550 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 20:32:19.092233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.092197    2550 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 20:32:19.092233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.092205    2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 20:32:19.092806 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.092796    2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 20:32:19.094390 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.094380    2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 20:32:19.097904 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.097892    2550 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 20:32:19.098027 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.098018    2550 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 20:32:19.098062 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.098050    2550 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 20:32:19.098092 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.098064    2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 20:32:19.098092 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.098082    2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 20:32:19.098145 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.098096    2550 kubelet.go:398] "Adding apiserver pod source"
Sep 09 20:32:19.098145 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.098108    2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 20:32:19.098580 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.098567    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 20:32:19.098644 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.098592    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 20:32:19.100730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.100699    2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 20:32:19.104278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104263    2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 20:32:19.104340 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104294    2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 20:32:19.104506 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104494    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 20:32:19.104558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104511    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 20:32:19.104558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104517    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 20:32:19.104558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104523    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 20:32:19.104558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104530    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 20:32:19.104558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104536    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 20:32:19.104828 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104818    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 20:32:19.104884 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104829    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 20:32:19.104884 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104852    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 20:32:19.104884 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104859    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 20:32:19.104884 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104884    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 20:32:19.104992 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.104894    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 20:32:19.105604 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.105592    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 20:32:19.105604 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.105602    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 20:32:19.107548 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.107531    2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-126.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 20:32:19.108085 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.108054    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-126.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 20:32:19.108153 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.108122    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 20:32:19.109500 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.109486    2550 server.go:1268] "Started kubelet"
Sep 09 20:32:19.109629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.109599    2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 20:32:19.111271 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.111238    2550 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 20:32:19.111846 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.109554    2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 20:32:19.112141 ip-10-0-140-126 systemd[1]: Started Kubernetes Kubelet.
Sep 09 20:32:19.113417 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.113397    2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 20:32:19.116657 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.116642    2550 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 20:32:19.116770 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.116748    2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 20:32:19.118637 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.117646    2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-126.ec2.internal.18d3c13bb0c9aff8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-126.ec2.internal,UID:ip-10-0-140-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-126.ec2.internal,},FirstTimestamp:2026-09-09 20:32:19.109457912 +0000 UTC m=+0.345424983,LastTimestamp:2026-09-09 20:32:19.109457912 +0000 UTC m=+0.345424983,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-126.ec2.internal,}"
Sep 09 20:32:19.120752 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.120731    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 20:32:19.121103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.121086    2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 20:32:19.121191 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.121147    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-126.ec2.internal\" not found"
Sep 09 20:32:19.121747 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.121721    2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 20:32:19.121831 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.121755    2550 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 20:32:19.121831 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.121766    2550 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 20:32:19.121940 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.121863    2550 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 20:32:19.121940 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.121888    2550 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 20:32:19.123487 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.123471    2550 factory.go:55] Registering systemd factory
Sep 09 20:32:19.123487 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.123488    2550 factory.go:223] Registration of the systemd container factory successfully
Sep 09 20:32:19.123774 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.123758    2550 factory.go:153] Registering CRI-O factory
Sep 09 20:32:19.123820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.123778    2550 factory.go:223] Registration of the crio container factory successfully
Sep 09 20:32:19.123888 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.123830    2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 20:32:19.123888 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.123849    2550 factory.go:103] Registering Raw factory
Sep 09 20:32:19.123888 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.123864    2550 manager.go:1196] Started watching for new ooms in manager
Sep 09 20:32:19.124278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.124265    2550 manager.go:349] Starting recovery of all containers
Sep 09 20:32:19.134409 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.134278    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 20:32:19.134409 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.134302    2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-126.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 20:32:19.135496 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.135483    2550 manager.go:354] Recovery completed
Sep 09 20:32:19.138546 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.138526    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rc6ql"
Sep 09 20:32:19.140421 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.140407    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:32:19.142187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.142169    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:32:19.142277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.142203    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:32:19.142277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.142226    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:32:19.144593 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.144573    2550 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 20:32:19.144593 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.144593    2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 20:32:19.144726 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.144612    2550 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 20:32:19.144854 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.144814    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:32:19.145636 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.145616    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rc6ql"
Sep 09 20:32:19.145952 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.145935    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:32:19.146016 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.145964    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:32:19.146016 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.145978    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:32:19.152895 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.152792    2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-126.ec2.internal.18d3c13bb2bd13ee  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-126.ec2.internal,UID:ip-10-0-140-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-126.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-126.ec2.internal,},FirstTimestamp:2026-09-09 20:32:19.142185966 +0000 UTC m=+0.378153037,LastTimestamp:2026-09-09 20:32:19.142185966 +0000 UTC m=+0.378153037,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-126.ec2.internal,}"
Sep 09 20:32:19.163803 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.163731    2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-126.ec2.internal.18d3c13bb2bd7110  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-126.ec2.internal,UID:ip-10-0-140-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-126.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-126.ec2.internal,},FirstTimestamp:2026-09-09 20:32:19.142209808 +0000 UTC m=+0.378176881,LastTimestamp:2026-09-09 20:32:19.142209808 +0000 UTC m=+0.378176881,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-126.ec2.internal,}"
Sep 09 20:32:19.175559 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.175495    2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-126.ec2.internal.18d3c13bb2bdc5d2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-126.ec2.internal,UID:ip-10-0-140-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-126.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-126.ec2.internal,},FirstTimestamp:2026-09-09 20:32:19.142231506 +0000 UTC m=+0.378198577,LastTimestamp:2026-09-09 20:32:19.142231506 +0000 UTC m=+0.378198577,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-126.ec2.internal,}"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.183925    2550 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-126.ec2.internal.18d3c13bb2bd13ee\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-126.ec2.internal.18d3c13bb2bd13ee  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-126.ec2.internal,UID:ip-10-0-140-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-126.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-126.ec2.internal,},FirstTimestamp:2026-09-09 20:32:19.142185966 +0000 UTC m=+0.378153037,LastTimestamp:2026-09-09 20:32:19.145950482 +0000 UTC m=+0.381917552,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-126.ec2.internal,}"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.186400    2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.187607    2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.187656    2550 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.187682    2550 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.187727    2550 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.193469    2550 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-126.ec2.internal.18d3c13bb2bd7110\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-126.ec2.internal.18d3c13bb2bd7110  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-126.ec2.internal,UID:ip-10-0-140-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-126.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-126.ec2.internal,},FirstTimestamp:2026-09-09 20:32:19.142209808 +0000 UTC m=+0.378176881,LastTimestamp:2026-09-09 20:32:19.1459714 +0000 UTC m=+0.381938474,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-126.ec2.internal,}"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.193585    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.199186    2550 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-126.ec2.internal.18d3c13bb2bdc5d2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-126.ec2.internal.18d3c13bb2bdc5d2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-126.ec2.internal,UID:ip-10-0-140-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-126.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-126.ec2.internal,},FirstTimestamp:2026-09-09 20:32:19.142231506 +0000 UTC m=+0.378198577,LastTimestamp:2026-09-09 20:32:19.14598204 +0000 UTC m=+0.381949111,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-126.ec2.internal,}"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.221399    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-126.ec2.internal\" not found"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.288174    2550 policy_none.go:49] "None policy: Start"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.288204    2550 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.288216    2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 20:32:19.288909 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.288655    2550 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Sep 09 20:32:19.289774 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.289694    2550 policy_none.go:47] "Start"
Sep 09 20:32:19.317779 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.317764    2550 manager.go:337] "Starting Device Plugin manager"
Sep 09 20:32:19.317862 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.317802    2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 20:32:19.317862 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.317811    2550 server.go:87] "Starting device plugin registration server"
Sep 09 20:32:19.318177 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.318166    2550 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 20:32:19.318210 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.318178    2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 20:32:19.318289 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.318265    2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 20:32:19.318386 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.318361    2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 20:32:19.318386 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.318372    2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 20:32:19.319674 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.319657    2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 20:32:19.319743 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.319700    2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-126.ec2.internal\" not found"
Sep 09 20:32:19.347704 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.347684    2550 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-126.ec2.internal\" not found" node="ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.419006 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.418979    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:32:19.419928 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.419909    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:32:19.420026 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.419944    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:32:19.420026 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.419958    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:32:19.420026 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.419986    2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.429394 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.429377    2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.429479 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.429399    2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-126.ec2.internal\": node \"ip-10-0-140-126.ec2.internal\" not found"
Sep 09 20:32:19.476241 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.476214    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-126.ec2.internal\" not found"
Sep 09 20:32:19.489313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.489290    2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal"]
Sep 09 20:32:19.489400 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.489386    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:32:19.490371 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.490356    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:32:19.490467 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.490388    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:32:19.490467 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.490403    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:32:19.491550 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.491536    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:32:19.491655 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.491642    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.491701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.491679    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:32:19.492291 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.492275    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:32:19.492352 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.492305    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:32:19.492352 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.492316    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:32:19.492411 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.492275    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:32:19.492411 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.492376    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:32:19.492411 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.492388    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:32:19.493387 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.493374    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.493458 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.493397    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:32:19.494084 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.494067    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:32:19.494145 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.494102    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:32:19.494145 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.494121    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:32:19.510390 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.510371    2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-126.ec2.internal\" not found" node="ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.514529 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.514515    2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-126.ec2.internal\" not found" node="ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.523927 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.523911    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e04d1c3fad7f79402f3ca02ea42b92c4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal\" (UID: \"e04d1c3fad7f79402f3ca02ea42b92c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.523927 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.523933    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbc80b7331391404881e6f559b693033-config\") pod \"kube-apiserver-proxy-ip-10-0-140-126.ec2.internal\" (UID: \"dbc80b7331391404881e6f559b693033\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.524039 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.523951    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e04d1c3fad7f79402f3ca02ea42b92c4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal\" (UID: \"e04d1c3fad7f79402f3ca02ea42b92c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.576798 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.576738    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-126.ec2.internal\" not found"
Sep 09 20:32:19.624392 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.624362    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e04d1c3fad7f79402f3ca02ea42b92c4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal\" (UID: \"e04d1c3fad7f79402f3ca02ea42b92c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.624522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.624398    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e04d1c3fad7f79402f3ca02ea42b92c4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal\" (UID: \"e04d1c3fad7f79402f3ca02ea42b92c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.624522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.624422    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbc80b7331391404881e6f559b693033-config\") pod \"kube-apiserver-proxy-ip-10-0-140-126.ec2.internal\" (UID: \"dbc80b7331391404881e6f559b693033\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.624522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.624464    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e04d1c3fad7f79402f3ca02ea42b92c4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal\" (UID: \"e04d1c3fad7f79402f3ca02ea42b92c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.624522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.624464    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e04d1c3fad7f79402f3ca02ea42b92c4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal\" (UID: \"e04d1c3fad7f79402f3ca02ea42b92c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.624522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.624467    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbc80b7331391404881e6f559b693033-config\") pod \"kube-apiserver-proxy-ip-10-0-140-126.ec2.internal\" (UID: \"dbc80b7331391404881e6f559b693033\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.677537 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.677493    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-126.ec2.internal\" not found"
Sep 09 20:32:19.778089 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.778061    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-126.ec2.internal\" not found"
Sep 09 20:32:19.814261 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.814238    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.818449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.818421    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:19.878665 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:19.878640    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-126.ec2.internal\" not found"
Sep 09 20:32:19.953504 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:19.953479    2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:32:20.021585 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.021560    2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:20.031801 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.031784    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 20:32:20.033815 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.033802    2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal"
Sep 09 20:32:20.049700 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.049679    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 20:32:20.050767 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.050754    2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 20:32:20.098417 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.098393    2550 apiserver.go:52] "Watching apiserver"
Sep 09 20:32:20.104410 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.104386    2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 20:32:20.105979 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.105956    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-b2m24","kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52","openshift-multus/multus-additional-cni-plugins-z5pdg","openshift-multus/network-metrics-daemon-jnrt5","openshift-network-diagnostics/network-check-target-xrfrn","openshift-ovn-kubernetes/ovnkube-node-bsmvv","openshift-cluster-node-tuning-operator/tuned-6mts9","openshift-dns/node-resolver-b5k8p","openshift-image-registry/node-ca-k6x8t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal","openshift-multus/multus-86drj","openshift-network-operator/iptables-alerter-pgz5h","kube-system/global-pull-secret-syncer-z64xx"]
Sep 09 20:32:20.107595 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.107580    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.108953 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.108930    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:20.109410 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.109396    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 20:32:20.109452 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.109418    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 20:32:20.109797 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.109781    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 20:32:20.109797 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.109789    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 20:32:20.110062 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.110046    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 20:32:20.110103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.110084    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 20:32:20.110103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.110096    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t6zxd\""
Sep 09 20:32:20.110285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.110272    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.111262 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.111237    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2r7j6\""
Sep 09 20:32:20.111337 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.111274    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:32:20.111337 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.111334    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 20:32:20.111415 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.111361    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 20:32:20.111813 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.111800    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.111928 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.111911    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 20:32:20.112164 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.112146    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 20:32:20.112230 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.112175    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 20:32:20.112230 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.112194    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ngvmm\""
Sep 09 20:32:20.113307 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.113292    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:20.113379 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.113356    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 20:32:20.113379 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.113361    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 20:32:20.113478 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.113357    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:20.113478 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.113448    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 20:32:20.113478 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.113439    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jls7v\""
Sep 09 20:32:20.113716 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.113699    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 20:32:20.113782 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.113769    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 20:32:20.114602 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.114590    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:20.114649 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.114635    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:20.115885 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.115859    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:20.117153 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.117139    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:20.117585 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.117565    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gnvbg\""
Sep 09 20:32:20.117666 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.117614    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 20:32:20.117666 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.117649    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 20:32:20.118608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.118591    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:20.118704 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.118686    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:20.118787 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.118770    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 20:32:20.118914 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.118893    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 20:32:20.119260 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.119245    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pgp6n\""
Sep 09 20:32:20.120125 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.120103    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:20.121657 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.121640    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 20:32:20.122413 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.121796    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.122413 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.121832    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 20:32:20.122413 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.121859    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 20:32:20.123611 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.123055    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-grplz\""
Sep 09 20:32:20.123611 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.123417    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 20:32:20.124099 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.124075    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:32:20.124211 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.124195    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6zh7f\""
Sep 09 20:32:20.125154 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.125129    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.126899 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.126864    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 20:32:20.126987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.126923    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-var-lib-kubelet\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.126987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.126956    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c659n\" (UniqueName: \"kubernetes.io/projected/922af2c0-0231-4892-8e2f-4d7aa4adbb29-kube-api-access-c659n\") pod \"iptables-alerter-pgz5h\" (UID: \"922af2c0-0231-4892-8e2f-4d7aa4adbb29\") " pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:20.127101 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.126990    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hkwnw\""
Sep 09 20:32:20.127101 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127036    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cf69c29-0898-41f4-8484-99472336cd97-cnibin\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.127101 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127093    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-socket-dir-parent\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.127241 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127114    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-run-ovn\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127241 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127162    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-env-overrides\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127241 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127186    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 20:32:20.127241 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127195    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-host\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.127380 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127265    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cf69c29-0898-41f4-8484-99472336cd97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.127600 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127586    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-run-openvswitch\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127651 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127606    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-log-socket\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127651 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127622    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-ovn-node-metrics-cert\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127651 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127637    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr28n\" (UniqueName: \"kubernetes.io/projected/fbfc5682-d88a-45cf-b4c2-aac06e054300-kube-api-access-dr28n\") pod \"node-ca-k6x8t\" (UID: \"fbfc5682-d88a-45cf-b4c2-aac06e054300\") " pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:20.127791 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127650    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-sysconfig\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.127791 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127689    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-etc-kubernetes\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.127791 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127742    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-ovnkube-config\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127791 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127767    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eeb00f2d-fb48-43d3-aee3-83a1fe8881c9-agent-certs\") pod \"konnectivity-agent-b2m24\" (UID: \"eeb00f2d-fb48-43d3-aee3-83a1fe8881c9\") " pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:20.127791 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127786    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fbfc5682-d88a-45cf-b4c2-aac06e054300-serviceca\") pod \"node-ca-k6x8t\" (UID: \"fbfc5682-d88a-45cf-b4c2-aac06e054300\") " pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:20.127958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127805    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-cni-dir\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.127958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127820    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-var-lib-cni-multus\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.127958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127834    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-kubelet\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127888    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-slash\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127907    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-run-netns\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127921    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-cni-bin\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127934    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-cni-netd\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.127958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127951    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pxbz\" (UniqueName: \"kubernetes.io/projected/ad1a3564-db90-40a2-9318-bef1ab32346b-kube-api-access-6pxbz\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.127983    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-var-lib-openvswitch\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128026    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-etc-openvswitch\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128041    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxk9n\" (UniqueName: \"kubernetes.io/projected/80ef3ff3-ff6f-48bc-929b-469f7a906385-kube-api-access-hxk9n\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128070    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cf69c29-0898-41f4-8484-99472336cd97-cni-binary-copy\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128112    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-var-lib-cni-bin\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128140    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-registration-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128163    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-lib-modules\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128189    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad1a3564-db90-40a2-9318-bef1ab32346b-cni-binary-copy\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128211    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-device-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128238    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-sysctl-conf\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128261    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:20.128285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128287    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-systemd-units\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128309    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128332    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-etc-selinux\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128352    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-sysctl-d\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128376    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-ovnkube-script-lib\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128453    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eca0688d-9168-4fe9-8ee9-4db243c38e7d-kubelet-config\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128481    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-system-cni-dir\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128500    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-run-k8s-cni-cncf-io\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128515    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9cf69c29-0898-41f4-8484-99472336cd97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128528    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7r5d\" (UniqueName: \"kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d\") pod \"network-check-target-xrfrn\" (UID: \"f28e9631-b9b8-4f66-b180-04b10282b574\") " pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128541    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-run-multus-certs\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128558    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fbfc5682-d88a-45cf-b4c2-aac06e054300-host\") pod \"node-ca-k6x8t\" (UID: \"fbfc5682-d88a-45cf-b4c2-aac06e054300\") " pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128574    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-tuned\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128587    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/922af2c0-0231-4892-8e2f-4d7aa4adbb29-host-slash\") pod \"iptables-alerter-pgz5h\" (UID: \"922af2c0-0231-4892-8e2f-4d7aa4adbb29\") " pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128601    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9cf69c29-0898-41f4-8484-99472336cd97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128621    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwn4j\" (UniqueName: \"kubernetes.io/projected/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-kube-api-access-dwn4j\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.128812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128646    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128666    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsdzk\" (UniqueName: \"kubernetes.io/projected/364732c6-5a16-44a2-afcf-9aaa6038c1f4-kube-api-access-qsdzk\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128693    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8f54dbf2-1cc5-4c0e-91c0-8dc53551965e-hosts-file\") pod \"node-resolver-b5k8p\" (UID: \"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e\") " pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128725    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/922af2c0-0231-4892-8e2f-4d7aa4adbb29-iptables-alerter-script\") pod \"iptables-alerter-pgz5h\" (UID: \"922af2c0-0231-4892-8e2f-4d7aa4adbb29\") " pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128742    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm98b\" (UniqueName: \"kubernetes.io/projected/9cf69c29-0898-41f4-8484-99472336cd97-kube-api-access-xm98b\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128755    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-cnibin\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128774    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-run-netns\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128804    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-conf-dir\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128817    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-run-systemd\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128831    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-kubernetes\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128862    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cf69c29-0898-41f4-8484-99472336cd97-system-cni-dir\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128902    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128917    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eeb00f2d-fb48-43d3-aee3-83a1fe8881c9-konnectivity-ca\") pod \"konnectivity-agent-b2m24\" (UID: \"eeb00f2d-fb48-43d3-aee3-83a1fe8881c9\") " pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128929    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-sys\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128941    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80ef3ff3-ff6f-48bc-929b-469f7a906385-tmp\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128968    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-socket-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.129278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.128986    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f54dbf2-1cc5-4c0e-91c0-8dc53551965e-tmp-dir\") pod \"node-resolver-b5k8p\" (UID: \"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e\") " pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129007    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-var-lib-kubelet\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129024    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-node-log\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129046    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dnsw\" (UniqueName: \"kubernetes.io/projected/3e1469ff-d369-404c-9583-36de8f7441bd-kube-api-access-8dnsw\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129066    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcz6v\" (UniqueName: \"kubernetes.io/projected/8f54dbf2-1cc5-4c0e-91c0-8dc53551965e-kube-api-access-fcz6v\") pod \"node-resolver-b5k8p\" (UID: \"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e\") " pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129079    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-run\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129093    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eca0688d-9168-4fe9-8ee9-4db243c38e7d-dbus\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129105    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-hostroot\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129118    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-daemon-config\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129131    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-run-ovn-kubernetes\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129143    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-sys-fs\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129159    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-modprobe-d\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.129717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.129177    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-systemd\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.131353 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.131335    2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 20:32:20.157112 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.157083    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wbt6d"
Sep 09 20:32:20.166035 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.166001    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wbt6d"
Sep 09 20:32:20.222572 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.222544    2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 20:32:20.230255 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230233    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cf69c29-0898-41f4-8484-99472336cd97-cnibin\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.230378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230261    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-socket-dir-parent\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.230378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230276    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-run-ovn\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.230378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230290    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-env-overrides\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.230378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230304    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-host\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.230378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230355    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-host\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.230378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230357    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cf69c29-0898-41f4-8484-99472336cd97-cnibin\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.230378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230371    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-socket-dir-parent\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.230378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230359    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-run-ovn\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230383    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cf69c29-0898-41f4-8484-99472336cd97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230414    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-run-openvswitch\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230438    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-log-socket\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230462    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-ovn-node-metrics-cert\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230477    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-run-openvswitch\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230484    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr28n\" (UniqueName: \"kubernetes.io/projected/fbfc5682-d88a-45cf-b4c2-aac06e054300-kube-api-access-dr28n\") pod \"node-ca-k6x8t\" (UID: \"fbfc5682-d88a-45cf-b4c2-aac06e054300\") " pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230517    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-sysconfig\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230537    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cf69c29-0898-41f4-8484-99472336cd97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230542    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-etc-kubernetes\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230552    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-log-socket\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230588    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-ovnkube-config\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230622    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-etc-kubernetes\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230629    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eeb00f2d-fb48-43d3-aee3-83a1fe8881c9-agent-certs\") pod \"konnectivity-agent-b2m24\" (UID: \"eeb00f2d-fb48-43d3-aee3-83a1fe8881c9\") " pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230653    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fbfc5682-d88a-45cf-b4c2-aac06e054300-serviceca\") pod \"node-ca-k6x8t\" (UID: \"fbfc5682-d88a-45cf-b4c2-aac06e054300\") " pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230677    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-cni-dir\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230689    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-sysconfig\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230706    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-var-lib-cni-multus\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.230820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230742    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-kubelet\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230774    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-slash\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230799    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-run-netns\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230825    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-cni-bin\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230885    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-cni-netd\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230914    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pxbz\" (UniqueName: \"kubernetes.io/projected/ad1a3564-db90-40a2-9318-bef1ab32346b-kube-api-access-6pxbz\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230943    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-var-lib-openvswitch\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230961    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-env-overrides\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230969    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-etc-openvswitch\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.230998    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxk9n\" (UniqueName: \"kubernetes.io/projected/80ef3ff3-ff6f-48bc-929b-469f7a906385-kube-api-access-hxk9n\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231024    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cf69c29-0898-41f4-8484-99472336cd97-cni-binary-copy\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231044    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-run-netns\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231072    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-var-lib-cni-bin\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231103    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-registration-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231151    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-lib-modules\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231159    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-var-lib-cni-multus\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231177    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad1a3564-db90-40a2-9318-bef1ab32346b-cni-binary-copy\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.231727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231201    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-device-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231210    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-kubelet\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231228    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-sysctl-conf\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231257    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-slash\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231273    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231303    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-systemd-units\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231326    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231350    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-etc-selinux\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231377    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-sysctl-d\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231401    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-ovnkube-script-lib\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231428    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eca0688d-9168-4fe9-8ee9-4db243c38e7d-kubelet-config\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231410    2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231564    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-system-cni-dir\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231595    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-ovnkube-config\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231618    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-cni-bin\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231682    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-cni-netd\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231734    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fbfc5682-d88a-45cf-b4c2-aac06e054300-serviceca\") pod \"node-ca-k6x8t\" (UID: \"fbfc5682-d88a-45cf-b4c2-aac06e054300\") " pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231734    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-device-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.232563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231839    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-sysctl-conf\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.233288 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231825    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-etc-selinux\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.233288 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231974    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-systemd-units\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.233288 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.232030    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.233288 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.232819    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-ovnkube-script-lib\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.233288 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.233087    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-var-lib-openvswitch\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.233288 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.233182    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-etc-openvswitch\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.233509 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.233497    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad1a3564-db90-40a2-9318-bef1ab32346b-cni-binary-copy\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.233795 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231455    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-system-cni-dir\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.233795 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.233767    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eca0688d-9168-4fe9-8ee9-4db243c38e7d-kubelet-config\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:20.233929 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.233865    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-registration-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.233988 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.233946    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-var-lib-cni-bin\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.233988 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.231116    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-cni-dir\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.234089 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.233964    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cf69c29-0898-41f4-8484-99472336cd97-cni-binary-copy\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.234089 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.233994    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-run-k8s-cni-cncf-io\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.234089 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.234056    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9cf69c29-0898-41f4-8484-99472336cd97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.234089 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.234068    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-sysctl-d\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.234267 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.234102    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7r5d\" (UniqueName: \"kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d\") pod \"network-check-target-xrfrn\" (UID: \"f28e9631-b9b8-4f66-b180-04b10282b574\") " pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:20.235128 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.234498    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:20.235128 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.234699    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret podName:eca0688d-9168-4fe9-8ee9-4db243c38e7d nodeName:}" failed. No retries permitted until 2026-09-09 20:32:20.734679528 +0000 UTC m=+1.970646603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret") pod "global-pull-secret-syncer-z64xx" (UID: "eca0688d-9168-4fe9-8ee9-4db243c38e7d") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:20.235128 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.234725    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9cf69c29-0898-41f4-8484-99472336cd97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.235128 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.234782    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-lib-modules\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.235128 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.234849    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-run-multus-certs\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.235128 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.234895    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fbfc5682-d88a-45cf-b4c2-aac06e054300-host\") pod \"node-ca-k6x8t\" (UID: \"fbfc5682-d88a-45cf-b4c2-aac06e054300\") " pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:20.235128 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.234928    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-run-multus-certs\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.235128 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.234930    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-tuned\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.235128 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235020    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fbfc5682-d88a-45cf-b4c2-aac06e054300-host\") pod \"node-ca-k6x8t\" (UID: \"fbfc5682-d88a-45cf-b4c2-aac06e054300\") " pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:20.235128 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235084    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-run-k8s-cni-cncf-io\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.235629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235135    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/922af2c0-0231-4892-8e2f-4d7aa4adbb29-host-slash\") pod \"iptables-alerter-pgz5h\" (UID: \"922af2c0-0231-4892-8e2f-4d7aa4adbb29\") " pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:20.235629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235169    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9cf69c29-0898-41f4-8484-99472336cd97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.235629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235243    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwn4j\" (UniqueName: \"kubernetes.io/projected/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-kube-api-access-dwn4j\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.235629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235284    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:20.235629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235312    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsdzk\" (UniqueName: \"kubernetes.io/projected/364732c6-5a16-44a2-afcf-9aaa6038c1f4-kube-api-access-qsdzk\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:20.235629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235367    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8f54dbf2-1cc5-4c0e-91c0-8dc53551965e-hosts-file\") pod \"node-resolver-b5k8p\" (UID: \"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e\") " pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:20.235629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235468    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8f54dbf2-1cc5-4c0e-91c0-8dc53551965e-hosts-file\") pod \"node-resolver-b5k8p\" (UID: \"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e\") " pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:20.235629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235476    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-ovn-node-metrics-cert\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.235580    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235671    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/922af2c0-0231-4892-8e2f-4d7aa4adbb29-host-slash\") pod \"iptables-alerter-pgz5h\" (UID: \"922af2c0-0231-4892-8e2f-4d7aa4adbb29\") " pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235702    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/922af2c0-0231-4892-8e2f-4d7aa4adbb29-iptables-alerter-script\") pod \"iptables-alerter-pgz5h\" (UID: \"922af2c0-0231-4892-8e2f-4d7aa4adbb29\") " pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235733    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm98b\" (UniqueName: \"kubernetes.io/projected/9cf69c29-0898-41f4-8484-99472336cd97-kube-api-access-xm98b\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235772    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-cnibin\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235803    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-run-netns\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235830    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-conf-dir\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235860    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-run-systemd\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235897    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-kubernetes\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235931    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cf69c29-0898-41f4-8484-99472336cd97-system-cni-dir\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235968    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.235996    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eeb00f2d-fb48-43d3-aee3-83a1fe8881c9-konnectivity-ca\") pod \"konnectivity-agent-b2m24\" (UID: \"eeb00f2d-fb48-43d3-aee3-83a1fe8881c9\") " pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:20.236025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236020    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-sys\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236052    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80ef3ff3-ff6f-48bc-929b-469f7a906385-tmp\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236080    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-socket-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236110    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f54dbf2-1cc5-4c0e-91c0-8dc53551965e-tmp-dir\") pod \"node-resolver-b5k8p\" (UID: \"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e\") " pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236138    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-var-lib-kubelet\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236161    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-node-log\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236193    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dnsw\" (UniqueName: \"kubernetes.io/projected/3e1469ff-d369-404c-9583-36de8f7441bd-kube-api-access-8dnsw\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236223    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcz6v\" (UniqueName: \"kubernetes.io/projected/8f54dbf2-1cc5-4c0e-91c0-8dc53551965e-kube-api-access-fcz6v\") pod \"node-resolver-b5k8p\" (UID: \"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e\") " pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236250    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-run\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236278    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eca0688d-9168-4fe9-8ee9-4db243c38e7d-dbus\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236302    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-hostroot\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236333    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-daemon-config\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236362    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-run-ovn-kubernetes\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236396    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-sys-fs\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236424    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-modprobe-d\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236453    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-systemd\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236477    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-var-lib-kubelet\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.236608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236505    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c659n\" (UniqueName: \"kubernetes.io/projected/922af2c0-0231-4892-8e2f-4d7aa4adbb29-kube-api-access-c659n\") pod \"iptables-alerter-pgz5h\" (UID: \"922af2c0-0231-4892-8e2f-4d7aa4adbb29\") " pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:20.237409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236631    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9cf69c29-0898-41f4-8484-99472336cd97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.237409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236859    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-kubernetes\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.237409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236925    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cf69c29-0898-41f4-8484-99472336cd97-system-cni-dir\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.237409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236969    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.237409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.237104    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-cnibin\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.237409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.237309    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-sys\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.237693 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.237590    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-tuned\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.237693 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.237628    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eeb00f2d-fb48-43d3-aee3-83a1fe8881c9-konnectivity-ca\") pod \"konnectivity-agent-b2m24\" (UID: \"eeb00f2d-fb48-43d3-aee3-83a1fe8881c9\") " pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:20.237693 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.237666    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-run-netns\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.237835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.237731    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-run\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.237914 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.237836    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs podName:364732c6-5a16-44a2-afcf-9aaa6038c1f4 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:20.737297003 +0000 UTC m=+1.973264079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs") pod "network-metrics-daemon-jnrt5" (UID: "364732c6-5a16-44a2-afcf-9aaa6038c1f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:20.237975 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.237952    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eca0688d-9168-4fe9-8ee9-4db243c38e7d-dbus\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:20.237975 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.237959    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-host-var-lib-kubelet\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.238067 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238048    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-node-log\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.238133 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238113    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-hostroot\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.238228 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238210    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-socket-dir\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.238283 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238236    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr28n\" (UniqueName: \"kubernetes.io/projected/fbfc5682-d88a-45cf-b4c2-aac06e054300-kube-api-access-dr28n\") pod \"node-ca-k6x8t\" (UID: \"fbfc5682-d88a-45cf-b4c2-aac06e054300\") " pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:20.238435 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238414    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-daemon-config\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.238502 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238490    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-host-run-ovn-kubernetes\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.238564 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.236160    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-run-systemd\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.238564 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238549    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e1469ff-d369-404c-9583-36de8f7441bd-sys-fs\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:20.238667 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238616    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad1a3564-db90-40a2-9318-bef1ab32346b-multus-conf-dir\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.238667 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238645    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/922af2c0-0231-4892-8e2f-4d7aa4adbb29-iptables-alerter-script\") pod \"iptables-alerter-pgz5h\" (UID: \"922af2c0-0231-4892-8e2f-4d7aa4adbb29\") " pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:20.238764 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238679    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-systemd\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.238764 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238741    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-var-lib-kubelet\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.238858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238768    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80ef3ff3-ff6f-48bc-929b-469f7a906385-etc-modprobe-d\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.238858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.238790    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f54dbf2-1cc5-4c0e-91c0-8dc53551965e-tmp-dir\") pod \"node-resolver-b5k8p\" (UID: \"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e\") " pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:20.239452 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.239435    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eeb00f2d-fb48-43d3-aee3-83a1fe8881c9-agent-certs\") pod \"konnectivity-agent-b2m24\" (UID: \"eeb00f2d-fb48-43d3-aee3-83a1fe8881c9\") " pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:20.239791 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.239777    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80ef3ff3-ff6f-48bc-929b-469f7a906385-tmp\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.241358 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.241341    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:32:20.241435 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.241365    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:32:20.241435 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.241378    2550 projected.go:196] Error preparing data for projected volume kube-api-access-l7r5d for pod openshift-network-diagnostics/network-check-target-xrfrn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:20.241435 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:20.241421    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d podName:f28e9631-b9b8-4f66-b180-04b10282b574 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:20.741409333 +0000 UTC m=+1.977376395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l7r5d" (UniqueName: "kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d") pod "network-check-target-xrfrn" (UID: "f28e9631-b9b8-4f66-b180-04b10282b574") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:20.243267 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.243243    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pxbz\" (UniqueName: \"kubernetes.io/projected/ad1a3564-db90-40a2-9318-bef1ab32346b-kube-api-access-6pxbz\") pod \"multus-86drj\" (UID: \"ad1a3564-db90-40a2-9318-bef1ab32346b\") " pod="openshift-multus/multus-86drj"
Sep 09 20:32:20.243367 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.243326    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxk9n\" (UniqueName: \"kubernetes.io/projected/80ef3ff3-ff6f-48bc-929b-469f7a906385-kube-api-access-hxk9n\") pod \"tuned-6mts9\" (UID: \"80ef3ff3-ff6f-48bc-929b-469f7a906385\") " pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:20.244882 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.244853    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwn4j\" (UniqueName: \"kubernetes.io/projected/d9d5ede3-b0a8-40ba-9856-0d78fab8f78b-kube-api-access-dwn4j\") pod \"ovnkube-node-bsmvv\" (UID: \"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:20.245992 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.245974    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c659n\" (UniqueName: \"kubernetes.io/projected/922af2c0-0231-4892-8e2f-4d7aa4adbb29-kube-api-access-c659n\") pod \"iptables-alerter-pgz5h\" (UID: \"922af2c0-0231-4892-8e2f-4d7aa4adbb29\") " pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:20.246291 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.246270    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm98b\" (UniqueName: \"kubernetes.io/projected/9cf69c29-0898-41f4-8484-99472336cd97-kube-api-access-xm98b\") pod \"multus-additional-cni-plugins-z5pdg\" (UID: \"9cf69c29-0898-41f4-8484-99472336cd97\") " pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:20.246368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.246305    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsdzk\" (UniqueName: \"kubernetes.io/projected/364732c6-5a16-44a2-afcf-9aaa6038c1f4-kube-api-access-qsdzk\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:20.246488 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.246470    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcz6v\" (UniqueName: \"kubernetes.io/projected/8f54dbf2-1cc5-4c0e-91c0-8dc53551965e-kube-api-access-fcz6v\") pod \"node-resolver-b5k8p\" (UID: \"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e\") " pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:20.246525 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:20.246493    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dnsw\" (UniqueName: \"kubernetes.io/projected/3e1469ff-d369-404c-9583-36de8f7441bd-kube-api-access-8dnsw\") pod \"aws-ebs-csi-driver-node-xnw52\" (UID: \"3e1469ff-d369-404c-9583-36de8f7441bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:21.359146 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.359122    2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:32:21.361654 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.361630    2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:32:21.374701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.374640    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 20:27:20 +0000 UTC" deadline="2028-04-25 00:31:58.276480105 +0000 UTC"
Sep 09 20:32:21.374701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.374695    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14235h59m36.901789109s"
Sep 09 20:32:21.448145 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.448122    2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:32:21.512148 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:21.512121    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc80b7331391404881e6f559b693033.slice/crio-f14bf2c13678ce8eab1ce337d2d3df62f5555f680f4b6d38d1969aa066ca0159 WatchSource:0}: Error finding container f14bf2c13678ce8eab1ce337d2d3df62f5555f680f4b6d38d1969aa066ca0159: Status 404 returned error can't find the container with id f14bf2c13678ce8eab1ce337d2d3df62f5555f680f4b6d38d1969aa066ca0159
Sep 09 20:32:21.512470 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:21.512447    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode04d1c3fad7f79402f3ca02ea42b92c4.slice/crio-fce7cb76bece9144f5fbb6391df995029e3bb77e9cec115fb6ab79272951f9e3 WatchSource:0}: Error finding container fce7cb76bece9144f5fbb6391df995029e3bb77e9cec115fb6ab79272951f9e3: Status 404 returned error can't find the container with id fce7cb76bece9144f5fbb6391df995029e3bb77e9cec115fb6ab79272951f9e3
Sep 09 20:32:21.515527 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.515513    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:32:21.523805 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.523784    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:21.531144 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:21.531126    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d5ede3_b0a8_40ba_9856_0d78fab8f78b.slice/crio-fae05d43b62011fde763d933a27997634bb03ffa8ea70daabd44aa5677ae8014 WatchSource:0}: Error finding container fae05d43b62011fde763d933a27997634bb03ffa8ea70daabd44aa5677ae8014: Status 404 returned error can't find the container with id fae05d43b62011fde763d933a27997634bb03ffa8ea70daabd44aa5677ae8014
Sep 09 20:32:21.546747 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.546729    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pgz5h"
Sep 09 20:32:21.553997 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:21.553852    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922af2c0_0231_4892_8e2f_4d7aa4adbb29.slice/crio-5205a97d4e657652df7e08d663473c7942970f9185d46ddee0929ce4d3cf043e WatchSource:0}: Error finding container 5205a97d4e657652df7e08d663473c7942970f9185d46ddee0929ce4d3cf043e: Status 404 returned error can't find the container with id 5205a97d4e657652df7e08d663473c7942970f9185d46ddee0929ce4d3cf043e
Sep 09 20:32:21.567557 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.567533    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52"
Sep 09 20:32:21.571256 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.571233    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z5pdg"
Sep 09 20:32:21.574192 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:21.574157    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e1469ff_d369_404c_9583_36de8f7441bd.slice/crio-88ec78ca90aaba8bda8fe9966a5383a5fa58a77b28203c4e6e54a3bc42e9c9d2 WatchSource:0}: Error finding container 88ec78ca90aaba8bda8fe9966a5383a5fa58a77b28203c4e6e54a3bc42e9c9d2: Status 404 returned error can't find the container with id 88ec78ca90aaba8bda8fe9966a5383a5fa58a77b28203c4e6e54a3bc42e9c9d2
Sep 09 20:32:21.578606 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:21.578590    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf69c29_0898_41f4_8484_99472336cd97.slice/crio-66b44dde643a38dc94f82dfc435119a46bd4d9f6f82e213fe4a2c28b448ff495 WatchSource:0}: Error finding container 66b44dde643a38dc94f82dfc435119a46bd4d9f6f82e213fe4a2c28b448ff495: Status 404 returned error can't find the container with id 66b44dde643a38dc94f82dfc435119a46bd4d9f6f82e213fe4a2c28b448ff495
Sep 09 20:32:21.586791 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.586773    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b5k8p"
Sep 09 20:32:21.593904 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:21.593888    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f54dbf2_1cc5_4c0e_91c0_8dc53551965e.slice/crio-eccfbcf1ba95488e128f4d61e5c34cce3fa5fca81145c46a0e4447c0f6e32eb1 WatchSource:0}: Error finding container eccfbcf1ba95488e128f4d61e5c34cce3fa5fca81145c46a0e4447c0f6e32eb1: Status 404 returned error can't find the container with id eccfbcf1ba95488e128f4d61e5c34cce3fa5fca81145c46a0e4447c0f6e32eb1
Sep 09 20:32:21.595865 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.595837    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:21.602012 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:21.601992    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb00f2d_fb48_43d3_aee3_83a1fe8881c9.slice/crio-91dfd86c01570606bb9b107d68bbcfff9c2aa3b5b72cffee3de6c660a543d37c WatchSource:0}: Error finding container 91dfd86c01570606bb9b107d68bbcfff9c2aa3b5b72cffee3de6c660a543d37c: Status 404 returned error can't find the container with id 91dfd86c01570606bb9b107d68bbcfff9c2aa3b5b72cffee3de6c660a543d37c
Sep 09 20:32:21.609517 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.609494    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k6x8t"
Sep 09 20:32:21.614252 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.614234    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6mts9"
Sep 09 20:32:21.617458 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:21.617364    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbfc5682_d88a_45cf_b4c2_aac06e054300.slice/crio-53aeda8c6e9fc58f925903f89d63b9fee14cca1fd78e027549c00fd19097ef2a WatchSource:0}: Error finding container 53aeda8c6e9fc58f925903f89d63b9fee14cca1fd78e027549c00fd19097ef2a: Status 404 returned error can't find the container with id 53aeda8c6e9fc58f925903f89d63b9fee14cca1fd78e027549c00fd19097ef2a
Sep 09 20:32:21.622150 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.622129    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-86drj"
Sep 09 20:32:21.622656 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:21.622638    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80ef3ff3_ff6f_48bc_929b_469f7a906385.slice/crio-1c33a97e61e66b034a2ad669f58760bfcc4733d1ebc1199eaaa6cf370d9d759f WatchSource:0}: Error finding container 1c33a97e61e66b034a2ad669f58760bfcc4733d1ebc1199eaaa6cf370d9d759f: Status 404 returned error can't find the container with id 1c33a97e61e66b034a2ad669f58760bfcc4733d1ebc1199eaaa6cf370d9d759f
Sep 09 20:32:21.630445 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:21.630425    2550 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-86drj_openshift-multus(ad1a3564-db90-40a2-9318-bef1ab32346b): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 09 20:32:21.630524 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:21.630456    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-86drj" podUID="ad1a3564-db90-40a2-9318-bef1ab32346b"
Sep 09 20:32:21.825608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.825534    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:21.825608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.825572    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:21.825805 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:21.825651    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:21.825805 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:21.825698    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:21.825805 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:21.825709    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs podName:364732c6-5a16-44a2-afcf-9aaa6038c1f4 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:22.825696645 +0000 UTC m=+2.976368965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs") pod "network-metrics-daemon-jnrt5" (UID: "364732c6-5a16-44a2-afcf-9aaa6038c1f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:21.825805 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:21.825752    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret podName:eca0688d-9168-4fe9-8ee9-4db243c38e7d nodeName:}" failed. No retries permitted until 2026-09-09 20:32:22.825736026 +0000 UTC m=+2.976408360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret") pod "global-pull-secret-syncer-z64xx" (UID: "eca0688d-9168-4fe9-8ee9-4db243c38e7d") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:21.926734 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:21.926712    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7r5d\" (UniqueName: \"kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d\") pod \"network-check-target-xrfrn\" (UID: \"f28e9631-b9b8-4f66-b180-04b10282b574\") " pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:21.926864 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:21.926844    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:32:21.926864 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:21.926859    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:32:21.926930 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:21.926868    2550 projected.go:196] Error preparing data for projected volume kube-api-access-l7r5d for pod openshift-network-diagnostics/network-check-target-xrfrn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:21.926930 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:21.926918    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d podName:f28e9631-b9b8-4f66-b180-04b10282b574 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:22.926898333 +0000 UTC m=+3.077570671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l7r5d" (UniqueName: "kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d") pod "network-check-target-xrfrn" (UID: "f28e9631-b9b8-4f66-b180-04b10282b574") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:22.253281 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.253185    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:27:21 +0000 UTC" deadline="2028-05-11 12:00:44.148380184 +0000 UTC"
Sep 09 20:32:22.253281 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.253246    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14631h28m21.895137971s"
Sep 09 20:32:22.276046 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.276016    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:22.276206 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:22.276155    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:22.284157 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.284039    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" event={"ID":"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b","Type":"ContainerStarted","Data":"fae05d43b62011fde763d933a27997634bb03ffa8ea70daabd44aa5677ae8014"}
Sep 09 20:32:22.293323 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.293257    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal" event={"ID":"dbc80b7331391404881e6f559b693033","Type":"ContainerStarted","Data":"f14bf2c13678ce8eab1ce337d2d3df62f5555f680f4b6d38d1969aa066ca0159"}
Sep 09 20:32:22.301609 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.301584    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal" event={"ID":"e04d1c3fad7f79402f3ca02ea42b92c4","Type":"ContainerStarted","Data":"fce7cb76bece9144f5fbb6391df995029e3bb77e9cec115fb6ab79272951f9e3"}
Sep 09 20:32:22.312961 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.312934    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-86drj" event={"ID":"ad1a3564-db90-40a2-9318-bef1ab32346b","Type":"ContainerStarted","Data":"776f4c16f6d5c3ca2cdcbf4f4798d239fe7c6a5c4f7661862e389a7d2fe9d3cf"}
Sep 09 20:32:22.316702 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:22.316543    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-86drj" podUID="ad1a3564-db90-40a2-9318-bef1ab32346b"
Sep 09 20:32:22.322034 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.321979    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b5k8p" event={"ID":"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e","Type":"ContainerStarted","Data":"eccfbcf1ba95488e128f4d61e5c34cce3fa5fca81145c46a0e4447c0f6e32eb1"}
Sep 09 20:32:22.327809 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.327732    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52" event={"ID":"3e1469ff-d369-404c-9583-36de8f7441bd","Type":"ContainerStarted","Data":"88ec78ca90aaba8bda8fe9966a5383a5fa58a77b28203c4e6e54a3bc42e9c9d2"}
Sep 09 20:32:22.334927 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.334902    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pgz5h" event={"ID":"922af2c0-0231-4892-8e2f-4d7aa4adbb29","Type":"ContainerStarted","Data":"5205a97d4e657652df7e08d663473c7942970f9185d46ddee0929ce4d3cf043e"}
Sep 09 20:32:22.344735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.344712    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6mts9" event={"ID":"80ef3ff3-ff6f-48bc-929b-469f7a906385","Type":"ContainerStarted","Data":"1c33a97e61e66b034a2ad669f58760bfcc4733d1ebc1199eaaa6cf370d9d759f"}
Sep 09 20:32:22.351032 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.351008    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k6x8t" event={"ID":"fbfc5682-d88a-45cf-b4c2-aac06e054300","Type":"ContainerStarted","Data":"53aeda8c6e9fc58f925903f89d63b9fee14cca1fd78e027549c00fd19097ef2a"}
Sep 09 20:32:22.356139 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.356085    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b2m24" event={"ID":"eeb00f2d-fb48-43d3-aee3-83a1fe8881c9","Type":"ContainerStarted","Data":"91dfd86c01570606bb9b107d68bbcfff9c2aa3b5b72cffee3de6c660a543d37c"}
Sep 09 20:32:22.363227 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.363204    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5pdg" event={"ID":"9cf69c29-0898-41f4-8484-99472336cd97","Type":"ContainerStarted","Data":"66b44dde643a38dc94f82dfc435119a46bd4d9f6f82e213fe4a2c28b448ff495"}
Sep 09 20:32:22.833404 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.833346    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:22.833572 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.833411    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:22.833572 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:22.833538    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:22.833672 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:22.833600    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs podName:364732c6-5a16-44a2-afcf-9aaa6038c1f4 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:24.833581584 +0000 UTC m=+4.984253910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs") pod "network-metrics-daemon-jnrt5" (UID: "364732c6-5a16-44a2-afcf-9aaa6038c1f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:22.833987 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:22.833971    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:22.834065 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:22.834025    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret podName:eca0688d-9168-4fe9-8ee9-4db243c38e7d nodeName:}" failed. No retries permitted until 2026-09-09 20:32:24.83400757 +0000 UTC m=+4.984679954 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret") pod "global-pull-secret-syncer-z64xx" (UID: "eca0688d-9168-4fe9-8ee9-4db243c38e7d") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:22.935104 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:22.934529    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7r5d\" (UniqueName: \"kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d\") pod \"network-check-target-xrfrn\" (UID: \"f28e9631-b9b8-4f66-b180-04b10282b574\") " pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:22.935104 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:22.934683    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:32:22.935104 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:22.934707    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:32:22.935104 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:22.934719    2550 projected.go:196] Error preparing data for projected volume kube-api-access-l7r5d for pod openshift-network-diagnostics/network-check-target-xrfrn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:22.935104 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:22.934784    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d podName:f28e9631-b9b8-4f66-b180-04b10282b574 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:24.934766506 +0000 UTC m=+5.085438831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l7r5d" (UniqueName: "kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d") pod "network-check-target-xrfrn" (UID: "f28e9631-b9b8-4f66-b180-04b10282b574") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:23.253603 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:23.253556    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:27:21 +0000 UTC" deadline="2028-06-11 02:49:43.414731044 +0000 UTC"
Sep 09 20:32:23.253603 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:23.253587    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15366h17m20.161147913s"
Sep 09 20:32:23.273923 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:23.273899    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:23.274064 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:23.274017    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:23.274939 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:23.274250    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:23.274939 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:23.274330    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:24.275569 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:24.275531    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:24.276012 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:24.275687    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:24.850268 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:24.850222    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:24.850461 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:24.850298    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:24.850461 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:24.850396    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:24.850461 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:24.850409    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:24.850461 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:24.850462    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret podName:eca0688d-9168-4fe9-8ee9-4db243c38e7d nodeName:}" failed. No retries permitted until 2026-09-09 20:32:28.850444615 +0000 UTC m=+9.001116948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret") pod "global-pull-secret-syncer-z64xx" (UID: "eca0688d-9168-4fe9-8ee9-4db243c38e7d") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:24.850668 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:24.850483    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs podName:364732c6-5a16-44a2-afcf-9aaa6038c1f4 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:28.850473275 +0000 UTC m=+9.001145597 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs") pod "network-metrics-daemon-jnrt5" (UID: "364732c6-5a16-44a2-afcf-9aaa6038c1f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:24.951406 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:24.951368    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7r5d\" (UniqueName: \"kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d\") pod \"network-check-target-xrfrn\" (UID: \"f28e9631-b9b8-4f66-b180-04b10282b574\") " pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:24.951592 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:24.951565    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:32:24.951592 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:24.951589    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:32:24.951698 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:24.951602    2550 projected.go:196] Error preparing data for projected volume kube-api-access-l7r5d for pod openshift-network-diagnostics/network-check-target-xrfrn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:24.951698 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:24.951668    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d podName:f28e9631-b9b8-4f66-b180-04b10282b574 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:28.951649386 +0000 UTC m=+9.102321725 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l7r5d" (UniqueName: "kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d") pod "network-check-target-xrfrn" (UID: "f28e9631-b9b8-4f66-b180-04b10282b574") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:25.273487 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:25.273284    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:25.273487 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:25.273324    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:25.273487 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:25.273445    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:25.273790 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:25.273606    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:26.274182 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:26.274134    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:26.274650 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:26.274293    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:27.274016 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:27.273979    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:27.274225 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:27.274112    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:27.274582 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:27.274284    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:27.274582 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:27.274397    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:28.273969 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:28.273933    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:28.274177 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:28.274078    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:28.880755 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:28.880718    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:28.881243 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:28.880780    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:28.881243 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:28.880863    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:28.881243 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:28.880910    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:28.881243 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:28.880949    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret podName:eca0688d-9168-4fe9-8ee9-4db243c38e7d nodeName:}" failed. No retries permitted until 2026-09-09 20:32:36.880927488 +0000 UTC m=+17.031599813 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret") pod "global-pull-secret-syncer-z64xx" (UID: "eca0688d-9168-4fe9-8ee9-4db243c38e7d") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:28.881243 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:28.880970    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs podName:364732c6-5a16-44a2-afcf-9aaa6038c1f4 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:36.880960778 +0000 UTC m=+17.031633099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs") pod "network-metrics-daemon-jnrt5" (UID: "364732c6-5a16-44a2-afcf-9aaa6038c1f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:28.981950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:28.981912    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7r5d\" (UniqueName: \"kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d\") pod \"network-check-target-xrfrn\" (UID: \"f28e9631-b9b8-4f66-b180-04b10282b574\") " pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:28.982146 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:28.982113    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:32:28.982146 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:28.982143    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:32:28.982338 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:28.982155    2550 projected.go:196] Error preparing data for projected volume kube-api-access-l7r5d for pod openshift-network-diagnostics/network-check-target-xrfrn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:28.982338 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:28.982237    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d podName:f28e9631-b9b8-4f66-b180-04b10282b574 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:36.982218581 +0000 UTC m=+17.132890906 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l7r5d" (UniqueName: "kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d") pod "network-check-target-xrfrn" (UID: "f28e9631-b9b8-4f66-b180-04b10282b574") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:29.273247 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:29.273209    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:29.273247 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:29.273234    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:29.273472 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:29.273347    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:29.273472 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:29.273441    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:30.274986 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:30.274952    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:30.275514 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:30.275090    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:31.273195 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:31.273148    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:31.273195 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:31.273198    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:31.273403 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:31.273285    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:31.273476 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:31.273400    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:32.273919 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:32.273890    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:32.274391 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:32.274016    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:33.274095 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:33.274063    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:33.274500 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:33.274207    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:33.274500 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:33.274261    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:33.274500 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:33.274382    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:34.274271 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:34.274186    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:34.274719 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:34.274348    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:35.274129 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.273946    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:35.274243 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.274007    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:35.274306 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:35.274256    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:35.275046 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:35.274386    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:35.403637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.403614    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:32:35.404062 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.404039    2550 generic.go:356] "Generic (PLEG): container finished" podID="d9d5ede3-b0a8-40ba-9856-0d78fab8f78b" containerID="27e13d8a0282a7e83957c0b5f925fd431fad161fa26fd0694191b27aa476c0b7" exitCode=1
Sep 09 20:32:35.404158 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.404105    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" event={"ID":"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b","Type":"ContainerStarted","Data":"f73d72080fb34857a9f2fb0307d5c3a6f77510fcc5148418fe71a767e539868f"}
Sep 09 20:32:35.404227 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.404216    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" event={"ID":"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b","Type":"ContainerDied","Data":"27e13d8a0282a7e83957c0b5f925fd431fad161fa26fd0694191b27aa476c0b7"}
Sep 09 20:32:35.404311 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.404269    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" event={"ID":"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b","Type":"ContainerStarted","Data":"f563e6cde9d11c9d49ed4ed6ca26e428c5577db33d3fca528a1dedcc2e301543"}
Sep 09 20:32:35.405952 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.405924    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal" event={"ID":"dbc80b7331391404881e6f559b693033","Type":"ContainerStarted","Data":"ae0bf020d82d50fe0fe83f47d16838f625955d9a53ff1b75ac395b34ee4f57a0"}
Sep 09 20:32:35.411054 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.411007    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6mts9" event={"ID":"80ef3ff3-ff6f-48bc-929b-469f7a906385","Type":"ContainerStarted","Data":"583e5e388e39c2e61906e72289acd27f8e51f5cb336c6fe3a94cae26ad101e6d"}
Sep 09 20:32:35.420288 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.420246    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-126.ec2.internal" podStartSLOduration=14.420230262 podStartE2EDuration="14.420230262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:32:35.419938982 +0000 UTC m=+15.570611326" watchObservedRunningTime="2026-09-09 20:32:35.420230262 +0000 UTC m=+15.570902607"
Sep 09 20:32:35.435955 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:35.435917    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6mts9" podStartSLOduration=2.294887135 podStartE2EDuration="15.435903589s" totalImagesPullingTime="13.141016454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:32:35.4354079 +0000 UTC m=+15.586080254" watchObservedRunningTime="2026-09-09 20:32:35.435903589 +0000 UTC m=+15.586575933"
Sep 09 20:32:36.273633 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.273436    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:36.273784 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:36.273738    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:36.415745 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.415654    2550 generic.go:356] "Generic (PLEG): container finished" podID="e04d1c3fad7f79402f3ca02ea42b92c4" containerID="88659c013417bd003ecfbf0a965e676468f8e097c00cd57f39fe180c8c4c212b" exitCode=0
Sep 09 20:32:36.416552 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.415754    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal" event={"ID":"e04d1c3fad7f79402f3ca02ea42b92c4","Type":"ContainerDied","Data":"88659c013417bd003ecfbf0a965e676468f8e097c00cd57f39fe180c8c4c212b"}
Sep 09 20:32:36.417679 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.417651    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b5k8p" event={"ID":"8f54dbf2-1cc5-4c0e-91c0-8dc53551965e","Type":"ContainerStarted","Data":"66118b23ee9d106b65c3100bd426b6a7cec03e5794d21e89d65e519d75e8d003"}
Sep 09 20:32:36.419795 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.419722    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52" event={"ID":"3e1469ff-d369-404c-9583-36de8f7441bd","Type":"ContainerStarted","Data":"29e3e0642929893702b0a7391e1ff44b8599671a9e5c95e6762817577e66ceca"}
Sep 09 20:32:36.421666 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.421644    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pgz5h" event={"ID":"922af2c0-0231-4892-8e2f-4d7aa4adbb29","Type":"ContainerStarted","Data":"e75b50be7bd9019d386fe07d86013320f7f01220787660b5a6a7d392f2c10d64"}
Sep 09 20:32:36.423470 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.423445    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k6x8t" event={"ID":"fbfc5682-d88a-45cf-b4c2-aac06e054300","Type":"ContainerStarted","Data":"0c15218f0c4f9b71de39f3f86f94845ccb37ab8c894f8d90706476c55b127693"}
Sep 09 20:32:36.425854 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.425828    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b2m24" event={"ID":"eeb00f2d-fb48-43d3-aee3-83a1fe8881c9","Type":"ContainerStarted","Data":"6cbb415524cc020511b851644708a1f844f8ffd5ec29120ddf6a9fdb309e7b3f"}
Sep 09 20:32:36.429690 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.429124    2550 generic.go:356] "Generic (PLEG): container finished" podID="9cf69c29-0898-41f4-8484-99472336cd97" containerID="64160f1da7a33801537a0ff91fd03338ab1da71f489f31f39ca46af7c68d53f1" exitCode=0
Sep 09 20:32:36.429690 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.429223    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5pdg" event={"ID":"9cf69c29-0898-41f4-8484-99472336cd97","Type":"ContainerDied","Data":"64160f1da7a33801537a0ff91fd03338ab1da71f489f31f39ca46af7c68d53f1"}
Sep 09 20:32:36.434690 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.434670    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:32:36.435458 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.435431    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" event={"ID":"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b","Type":"ContainerStarted","Data":"22e5771af31869fd316bf74ee5b8985f9425b78fb40400da2ca6e63020b5087e"}
Sep 09 20:32:36.435588 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.435514    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" event={"ID":"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b","Type":"ContainerStarted","Data":"dfb67f697b88078a40bb8f220ba6a2c79e41029fe9e9d8c88c68e163d937184a"}
Sep 09 20:32:36.435588 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.435565    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" event={"ID":"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b","Type":"ContainerStarted","Data":"de7e84517951119a381e790df5cc5a831ea4602edab0d9d58ec82e6cd472e3ae"}
Sep 09 20:32:36.444048 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.443684    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b5k8p" podStartSLOduration=3.288534282 podStartE2EDuration="16.443667221s" totalImagesPullingTime="13.155132939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:32:36.442791442 +0000 UTC m=+16.593463785" watchObservedRunningTime="2026-09-09 20:32:36.443667221 +0000 UTC m=+16.594339565"
Sep 09 20:32:36.457461 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.457413    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b2m24" podStartSLOduration=3.309601375 podStartE2EDuration="16.457395545s" totalImagesPullingTime="13.14779417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:32:36.456968625 +0000 UTC m=+16.607640970" watchObservedRunningTime="2026-09-09 20:32:36.457395545 +0000 UTC m=+16.608067889"
Sep 09 20:32:36.471525 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.471210    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k6x8t" podStartSLOduration=3.338694501 podStartE2EDuration="16.471194701s" totalImagesPullingTime="13.1325002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:32:36.470418935 +0000 UTC m=+16.621091279" watchObservedRunningTime="2026-09-09 20:32:36.471194701 +0000 UTC m=+16.621867045"
Sep 09 20:32:36.484808 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.484764    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pgz5h" podStartSLOduration=3.313519128 podStartE2EDuration="16.484747868s" totalImagesPullingTime="13.17122874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:32:36.483972423 +0000 UTC m=+16.634644769" watchObservedRunningTime="2026-09-09 20:32:36.484747868 +0000 UTC m=+16.635420217"
Sep 09 20:32:36.920819 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.920779    2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 20:32:36.942795 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.942772    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:36.942901 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:36.942817    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:36.942992 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:36.942917    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:36.942992 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:36.942921    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:36.942992 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:36.942981    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs podName:364732c6-5a16-44a2-afcf-9aaa6038c1f4 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:52.942963503 +0000 UTC m=+33.093635837 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs") pod "network-metrics-daemon-jnrt5" (UID: "364732c6-5a16-44a2-afcf-9aaa6038c1f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:32:36.943136 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:36.943001    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret podName:eca0688d-9168-4fe9-8ee9-4db243c38e7d nodeName:}" failed. No retries permitted until 2026-09-09 20:32:52.942990122 +0000 UTC m=+33.093662444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret") pod "global-pull-secret-syncer-z64xx" (UID: "eca0688d-9168-4fe9-8ee9-4db243c38e7d") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:32:37.043613 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:37.043573    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7r5d\" (UniqueName: \"kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d\") pod \"network-check-target-xrfrn\" (UID: \"f28e9631-b9b8-4f66-b180-04b10282b574\") " pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:37.043802 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:37.043780    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:32:37.043802 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:37.043803    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:32:37.043971 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:37.043816    2550 projected.go:196] Error preparing data for projected volume kube-api-access-l7r5d for pod openshift-network-diagnostics/network-check-target-xrfrn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:37.043971 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:37.043870    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d podName:f28e9631-b9b8-4f66-b180-04b10282b574 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:53.043854104 +0000 UTC m=+33.194526429 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l7r5d" (UniqueName: "kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d") pod "network-check-target-xrfrn" (UID: "f28e9631-b9b8-4f66-b180-04b10282b574") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:32:37.273467 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:37.273440    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:37.273641 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:37.273580    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:37.273704 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:37.273632    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:37.273767 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:37.273745    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:37.413096 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:37.412984    2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T20:32:36.920796732Z","UUID":"bfcbab72-8b4c-4129-8835-97ac297b281c","Handler":null,"Name":"","Endpoint":""}
Sep 09 20:32:37.415271 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:37.415240    2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 20:32:37.415391 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:37.415286    2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 20:32:37.440453 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:37.440422    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal" event={"ID":"e04d1c3fad7f79402f3ca02ea42b92c4","Type":"ContainerStarted","Data":"9e44afdf25dac1b02fc85590264b4397191f97b78a3030be17319b4e5994e6af"}
Sep 09 20:32:37.443286 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:37.443257    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52" event={"ID":"3e1469ff-d369-404c-9583-36de8f7441bd","Type":"ContainerStarted","Data":"8f6102b0879984bd874c46631fb25ad1e430d41fdfb78499c4420c6ff154f11a"}
Sep 09 20:32:37.455228 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:37.455179    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-126.ec2.internal" podStartSLOduration=16.45514738 podStartE2EDuration="16.45514738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:32:37.454694248 +0000 UTC m=+17.605366593" watchObservedRunningTime="2026-09-09 20:32:37.45514738 +0000 UTC m=+17.605819723"
Sep 09 20:32:38.274107 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:38.273887    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:38.274290 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:38.274257    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:38.449649 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:38.449626    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:32:38.450283 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:38.450184    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" event={"ID":"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b","Type":"ContainerStarted","Data":"4cbca92730313a9f933165e98540345fb9b84162607cea2f27cc842bf713919b"}
Sep 09 20:32:39.273351 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:39.273316    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:39.273641 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:39.273426    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:39.273739 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:39.273711    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:39.273856 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:39.273791    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:39.455757 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:39.455717    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52" event={"ID":"3e1469ff-d369-404c-9583-36de8f7441bd","Type":"ContainerStarted","Data":"5171f783171ea3df4a63e80c4df1070d8c5686a7d45f63f6ba1dc2384eeab575"}
Sep 09 20:32:39.474017 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:39.473089    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xnw52" podStartSLOduration=3.824904857 podStartE2EDuration="19.473069805s" totalImagesPullingTime="15.648164948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:32:39.472522972 +0000 UTC m=+19.623195316" watchObservedRunningTime="2026-09-09 20:32:39.473069805 +0000 UTC m=+19.623742148"
Sep 09 20:32:40.275176 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:40.275125    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:40.275331 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:40.275260    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:40.298821 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:40.298791    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:40.299436 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:40.299417    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:41.273509 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:41.273478    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:41.274038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:41.273477    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:41.274038 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:41.273604    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:41.274038 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:41.273671    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:42.273618 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:42.273376    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:42.274055 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:42.273728    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:42.465051 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:42.464996    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5pdg" event={"ID":"9cf69c29-0898-41f4-8484-99472336cd97","Type":"ContainerStarted","Data":"bd0c3ba5a78a617aab83b8ae7f9676667b3346bee2341fc97b5cd2ac099174a7"}
Sep 09 20:32:42.469111 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:42.469085    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:32:42.469611 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:42.469531    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" event={"ID":"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b","Type":"ContainerStarted","Data":"d6039a2adc5848ec31978cd8fcca561b683e8d29444c27211eaadcbef95dce8b"}
Sep 09 20:32:42.469869 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:42.469849    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:42.469985 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:42.469921    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:42.469985 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:42.469974    2550 scope.go:117] "RemoveContainer" containerID="27e13d8a0282a7e83957c0b5f925fd431fad161fa26fd0694191b27aa476c0b7"
Sep 09 20:32:42.486078 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:42.486054    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:42.486853 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:42.486835    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:42.535788 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:42.535706    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:43.273317 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:43.273205    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:43.273496 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:43.273332    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:43.273692 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:43.273674    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:43.274306 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:43.273772    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:43.524097 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:43.524013    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z64xx"]
Sep 09 20:32:43.524277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:43.524138    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:43.524335 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:43.524272    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:43.529232 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:43.529207    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jnrt5"]
Sep 09 20:32:43.529369 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:43.529320    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:43.529458 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:43.529431    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:43.536793 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:43.536771    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xrfrn"]
Sep 09 20:32:43.536891 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:43.536855    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:43.536943 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:43.536926    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:45.273210 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:45.273179    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:45.273810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:45.273180    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:45.273810 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:45.273294    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:45.273810 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:45.273374    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:45.273810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:45.273180    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:45.273810 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:45.273449    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:46.480096 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:46.479907    2550 generic.go:356] "Generic (PLEG): container finished" podID="9cf69c29-0898-41f4-8484-99472336cd97" containerID="bd0c3ba5a78a617aab83b8ae7f9676667b3346bee2341fc97b5cd2ac099174a7" exitCode=0
Sep 09 20:32:46.480481 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:46.479979    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5pdg" event={"ID":"9cf69c29-0898-41f4-8484-99472336cd97","Type":"ContainerDied","Data":"bd0c3ba5a78a617aab83b8ae7f9676667b3346bee2341fc97b5cd2ac099174a7"}
Sep 09 20:32:46.487457 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:46.487441    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:32:46.487858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:46.487837    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" event={"ID":"d9d5ede3-b0a8-40ba-9856-0d78fab8f78b","Type":"ContainerStarted","Data":"89c42d628adf6044c105838329ddab343a736656794f18c585abe1c859d3e2ab"}
Sep 09 20:32:46.489283 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:46.489265    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-86drj" event={"ID":"ad1a3564-db90-40a2-9318-bef1ab32346b","Type":"ContainerStarted","Data":"6dc69fe60b44e8f42b7b95ebf3a4c3cccbbc3daaa20342b369f3e770ecc28b97"}
Sep 09 20:32:46.497701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:46.497685    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv"
Sep 09 20:32:46.523448 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:46.523408    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bsmvv" podStartSLOduration=13.033136821 podStartE2EDuration="26.523391774s" totalImagesPullingTime="13.490254953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:32:46.523292449 +0000 UTC m=+26.673964792" watchObservedRunningTime="2026-09-09 20:32:46.523391774 +0000 UTC m=+26.674064114"
Sep 09 20:32:46.538916 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:46.538878    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-86drj" podStartSLOduration=2.241394794 podStartE2EDuration="26.538863239s" totalImagesPullingTime="24.297468445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-09 20:32:46.538377229 +0000 UTC m=+26.689049571" watchObservedRunningTime="2026-09-09 20:32:46.538863239 +0000 UTC m=+26.689535582"
Sep 09 20:32:47.274362 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.273959    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:47.274362 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.273973    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:47.274362 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:47.274122    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnrt5" podUID="364732c6-5a16-44a2-afcf-9aaa6038c1f4"
Sep 09 20:32:47.274362 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.273973    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:47.274362 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:47.274158    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z64xx" podUID="eca0688d-9168-4fe9-8ee9-4db243c38e7d"
Sep 09 20:32:47.274362 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:47.274252    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrfrn" podUID="f28e9631-b9b8-4f66-b180-04b10282b574"
Sep 09 20:32:47.493793 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.493713    2550 generic.go:356] "Generic (PLEG): container finished" podID="9cf69c29-0898-41f4-8484-99472336cd97" containerID="570a45e598b2ab459a5a7eba7b0c5fa85ed98f3a18d83d08e756ea9455e11628" exitCode=0
Sep 09 20:32:47.494123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.493786    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5pdg" event={"ID":"9cf69c29-0898-41f4-8484-99472336cd97","Type":"ContainerDied","Data":"570a45e598b2ab459a5a7eba7b0c5fa85ed98f3a18d83d08e756ea9455e11628"}
Sep 09 20:32:47.639767 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.639749    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-126.ec2.internal" event="NodeReady"
Sep 09 20:32:47.639889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.639850    2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 20:32:47.679988 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.677450    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-db66467bd-l6hrj"]
Sep 09 20:32:47.680457 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.680434    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"]
Sep 09 20:32:47.680581 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.680563    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.682071 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.682037    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"
Sep 09 20:32:47.682692 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.682392    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"]
Sep 09 20:32:47.684030 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.684015    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"]
Sep 09 20:32:47.684154 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.684139    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.685295 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.685274    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b4pxw\""
Sep 09 20:32:47.685433 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.685328    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 20:32:47.685518 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.685483    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 20:32:47.685584 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.685523    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-97ppf\""
Sep 09 20:32:47.685584 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.685483    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 20:32:47.685584 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.685565    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 20:32:47.685758 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.685635    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 20:32:47.685758 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.685666    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:47.685758 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.685731    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 20:32:47.685905 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.685830    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 09 20:32:47.686241 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.686229    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 09 20:32:47.686384 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.686344    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 09 20:32:47.686567 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.686552    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 09 20:32:47.686635 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.686591    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 09 20:32:47.688944 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.688927    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 09 20:32:47.693112 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.693075    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2jfmw"]
Sep 09 20:32:47.693361 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.693344    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 20:32:47.695027 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.695002    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.696827 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.696615    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"]
Sep 09 20:32:47.697119 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.697101    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 20:32:47.697225 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.697138    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 20:32:47.697469 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.697450    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db66467bd-l6hrj"]
Sep 09 20:32:47.697542 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.697499    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2sgfl\""
Sep 09 20:32:47.701704 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.701684    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"]
Sep 09 20:32:47.702249 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.702229    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-krxjb"]
Sep 09 20:32:47.703785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.703769    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-krxjb"
Sep 09 20:32:47.704669 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.704652    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"]
Sep 09 20:32:47.705449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.705431    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 20:32:47.705669 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.705652    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k7p7g\""
Sep 09 20:32:47.706253 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.706237    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 20:32:47.706340 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.706245    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 20:32:47.708659 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.708641    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2jfmw"]
Sep 09 20:32:47.716882 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.716859    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-krxjb"]
Sep 09 20:32:47.829246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829153    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shbjh\" (UniqueName: \"kubernetes.io/projected/af804414-5ad6-4aec-a47b-b2e47879ee82-kube-api-access-shbjh\") pod \"managed-serviceaccount-addon-agent-7fb65c7858-gdtjx\" (UID: \"af804414-5ad6-4aec-a47b-b2e47879ee82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"
Sep 09 20:32:47.829246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829209    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.829421 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829249    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef22136f-b71c-48bf-83ee-01d0881f8866-tmp-dir\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.829421 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829281    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0ee110d0-1ab4-4200-8c23-65c1f5d0584d-klusterlet-config\") pod \"klusterlet-addon-workmgr-84c589674c-qxtwl\" (UID: \"0ee110d0-1ab4-4200-8c23-65c1f5d0584d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:47.829421 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829319    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5sws\" (UniqueName: \"kubernetes.io/projected/0ee110d0-1ab4-4200-8c23-65c1f5d0584d-kube-api-access-x5sws\") pod \"klusterlet-addon-workmgr-84c589674c-qxtwl\" (UID: \"0ee110d0-1ab4-4200-8c23-65c1f5d0584d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:47.829421 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829358    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-image-registry-private-configuration\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.829421 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829383    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-installation-pull-secrets\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.829421 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829403    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-ca\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.829421 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829418    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ee110d0-1ab4-4200-8c23-65c1f5d0584d-tmp\") pod \"klusterlet-addon-workmgr-84c589674c-qxtwl\" (UID: \"0ee110d0-1ab4-4200-8c23-65c1f5d0584d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:47.829675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829442    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztr22\" (UniqueName: \"kubernetes.io/projected/38f31112-a172-4677-be02-544c74a0e0aa-kube-api-access-ztr22\") pod \"ingress-canary-krxjb\" (UID: \"38f31112-a172-4677-be02-544c74a0e0aa\") " pod="openshift-ingress-canary/ingress-canary-krxjb"
Sep 09 20:32:47.829675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829469    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.829675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829484    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b5b961c3-8408-4e91-ac9f-91b4c8baab66-ca-trust-extracted\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.829675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829504    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-certificates\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.829675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829522    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-bound-sa-token\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.829675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829538    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/af804414-5ad6-4aec-a47b-b2e47879ee82-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb65c7858-gdtjx\" (UID: \"af804414-5ad6-4aec-a47b-b2e47879ee82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"
Sep 09 20:32:47.829675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829554    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef22136f-b71c-48bf-83ee-01d0881f8866-config-volume\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.829675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829598    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-trusted-ca\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.829675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829617    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.829675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829632    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvwh7\" (UniqueName: \"kubernetes.io/projected/ef22136f-b71c-48bf-83ee-01d0881f8866-kube-api-access-rvwh7\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.829981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829681    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.829981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829705    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38f31112-a172-4677-be02-544c74a0e0aa-cert\") pod \"ingress-canary-krxjb\" (UID: \"38f31112-a172-4677-be02-544c74a0e0aa\") " pod="openshift-ingress-canary/ingress-canary-krxjb"
Sep 09 20:32:47.829981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829756    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-hub\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.829981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829777    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.829981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829803    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2kbc\" (UniqueName: \"kubernetes.io/projected/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-kube-api-access-h2kbc\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.829981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.829820    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57hpr\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-kube-api-access-57hpr\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.861179 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.861147    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:47.930288 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930264    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef22136f-b71c-48bf-83ee-01d0881f8866-config-volume\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.930375 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930301    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-trusted-ca\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.930375 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930319    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.930375 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930335    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvwh7\" (UniqueName: \"kubernetes.io/projected/ef22136f-b71c-48bf-83ee-01d0881f8866-kube-api-access-rvwh7\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.930375 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930357    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.930570 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930377    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38f31112-a172-4677-be02-544c74a0e0aa-cert\") pod \"ingress-canary-krxjb\" (UID: \"38f31112-a172-4677-be02-544c74a0e0aa\") " pod="openshift-ingress-canary/ingress-canary-krxjb"
Sep 09 20:32:47.930570 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930412    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-hub\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.930570 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930453    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.930716 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930697    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2kbc\" (UniqueName: \"kubernetes.io/projected/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-kube-api-access-h2kbc\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.930770 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930737    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57hpr\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-kube-api-access-57hpr\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.930857 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930776    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shbjh\" (UniqueName: \"kubernetes.io/projected/af804414-5ad6-4aec-a47b-b2e47879ee82-kube-api-access-shbjh\") pod \"managed-serviceaccount-addon-agent-7fb65c7858-gdtjx\" (UID: \"af804414-5ad6-4aec-a47b-b2e47879ee82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"
Sep 09 20:32:47.930857 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930798    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.930857 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930819    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef22136f-b71c-48bf-83ee-01d0881f8866-tmp-dir\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.930857 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930839    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0ee110d0-1ab4-4200-8c23-65c1f5d0584d-klusterlet-config\") pod \"klusterlet-addon-workmgr-84c589674c-qxtwl\" (UID: \"0ee110d0-1ab4-4200-8c23-65c1f5d0584d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:47.931044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930860    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5sws\" (UniqueName: \"kubernetes.io/projected/0ee110d0-1ab4-4200-8c23-65c1f5d0584d-kube-api-access-x5sws\") pod \"klusterlet-addon-workmgr-84c589674c-qxtwl\" (UID: \"0ee110d0-1ab4-4200-8c23-65c1f5d0584d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:47.931044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930889    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-image-registry-private-configuration\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.931044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930924    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-installation-pull-secrets\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.931044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930939    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef22136f-b71c-48bf-83ee-01d0881f8866-config-volume\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.931044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930952    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-ca\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.931044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.930973    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ee110d0-1ab4-4200-8c23-65c1f5d0584d-tmp\") pod \"klusterlet-addon-workmgr-84c589674c-qxtwl\" (UID: \"0ee110d0-1ab4-4200-8c23-65c1f5d0584d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:47.931044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931016    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztr22\" (UniqueName: \"kubernetes.io/projected/38f31112-a172-4677-be02-544c74a0e0aa-kube-api-access-ztr22\") pod \"ingress-canary-krxjb\" (UID: \"38f31112-a172-4677-be02-544c74a0e0aa\") " pod="openshift-ingress-canary/ingress-canary-krxjb"
Sep 09 20:32:47.931044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931020    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.931449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931064    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.931449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931090    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b5b961c3-8408-4e91-ac9f-91b4c8baab66-ca-trust-extracted\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.931449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931116    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-certificates\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.931449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931142    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-bound-sa-token\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.931449 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:47.931235    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:32:47.931449 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:47.931299    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls podName:ef22136f-b71c-48bf-83ee-01d0881f8866 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:48.43128147 +0000 UTC m=+28.581953807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls") pod "dns-default-2jfmw" (UID: "ef22136f-b71c-48bf-83ee-01d0881f8866") : secret "dns-default-metrics-tls" not found
Sep 09 20:32:47.931449 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:47.931326    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:32:47.931449 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:47.931338    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db66467bd-l6hrj: secret "image-registry-tls" not found
Sep 09 20:32:47.931449 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:47.931379    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls podName:b5b961c3-8408-4e91-ac9f-91b4c8baab66 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:48.431367124 +0000 UTC m=+28.582039457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls") pod "image-registry-db66467bd-l6hrj" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66") : secret "image-registry-tls" not found
Sep 09 20:32:47.931449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931439    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-trusted-ca\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.931941 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931482    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ee110d0-1ab4-4200-8c23-65c1f5d0584d-tmp\") pod \"klusterlet-addon-workmgr-84c589674c-qxtwl\" (UID: \"0ee110d0-1ab4-4200-8c23-65c1f5d0584d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:47.931941 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931532    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/af804414-5ad6-4aec-a47b-b2e47879ee82-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb65c7858-gdtjx\" (UID: \"af804414-5ad6-4aec-a47b-b2e47879ee82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"
Sep 09 20:32:47.931941 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931698    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef22136f-b71c-48bf-83ee-01d0881f8866-tmp-dir\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.931941 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931725    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b5b961c3-8408-4e91-ac9f-91b4c8baab66-ca-trust-extracted\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.931941 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.931864    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-certificates\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.936103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.935931    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-image-registry-private-configuration\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.936103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.936056    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.936103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.936064    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-hub\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.936344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.936206    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/af804414-5ad6-4aec-a47b-b2e47879ee82-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb65c7858-gdtjx\" (UID: \"af804414-5ad6-4aec-a47b-b2e47879ee82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"
Sep 09 20:32:47.936344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.936209    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0ee110d0-1ab4-4200-8c23-65c1f5d0584d-klusterlet-config\") pod \"klusterlet-addon-workmgr-84c589674c-qxtwl\" (UID: \"0ee110d0-1ab4-4200-8c23-65c1f5d0584d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:47.936344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.936309    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.936508 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.936470    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38f31112-a172-4677-be02-544c74a0e0aa-cert\") pod \"ingress-canary-krxjb\" (UID: \"38f31112-a172-4677-be02-544c74a0e0aa\") " pod="openshift-ingress-canary/ingress-canary-krxjb"
Sep 09 20:32:47.936700 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.936679    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-ca\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.937224 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.937202    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-installation-pull-secrets\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.942892 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.942860    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-bound-sa-token\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.943352 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.943328    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shbjh\" (UniqueName: \"kubernetes.io/projected/af804414-5ad6-4aec-a47b-b2e47879ee82-kube-api-access-shbjh\") pod \"managed-serviceaccount-addon-agent-7fb65c7858-gdtjx\" (UID: \"af804414-5ad6-4aec-a47b-b2e47879ee82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"
Sep 09 20:32:47.943823 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.943803    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5sws\" (UniqueName: \"kubernetes.io/projected/0ee110d0-1ab4-4200-8c23-65c1f5d0584d-kube-api-access-x5sws\") pod \"klusterlet-addon-workmgr-84c589674c-qxtwl\" (UID: \"0ee110d0-1ab4-4200-8c23-65c1f5d0584d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:47.943982 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.943962    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57hpr\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-kube-api-access-57hpr\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:47.944156 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.944136    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztr22\" (UniqueName: \"kubernetes.io/projected/38f31112-a172-4677-be02-544c74a0e0aa-kube-api-access-ztr22\") pod \"ingress-canary-krxjb\" (UID: \"38f31112-a172-4677-be02-544c74a0e0aa\") " pod="openshift-ingress-canary/ingress-canary-krxjb"
Sep 09 20:32:47.944156 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.944190    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvwh7\" (UniqueName: \"kubernetes.io/projected/ef22136f-b71c-48bf-83ee-01d0881f8866-kube-api-access-rvwh7\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:47.944156 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.944196    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2kbc\" (UniqueName: \"kubernetes.io/projected/bcde0f8e-13e1-4cb8-bb02-f963c359b8a2-kube-api-access-h2kbc\") pod \"cluster-proxy-proxy-agent-6f896dd9d7-v2nfc\" (UID: \"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:47.957750 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:47.957728    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:48.006436 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.006417    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"
Sep 09 20:32:48.013191 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.013121    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:32:48.032432 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.032400    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:32:48.045236 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.045157    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-krxjb"
Sep 09 20:32:48.301001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.300830    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx"]
Sep 09 20:32:48.304211 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.304152    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"]
Sep 09 20:32:48.307608 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:48.307320    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf804414_5ad6_4aec_a47b_b2e47879ee82.slice/crio-16fecff1a2e02127490fed2348591c616e48b5ce414e6285c0518bbe0e63011f WatchSource:0}: Error finding container 16fecff1a2e02127490fed2348591c616e48b5ce414e6285c0518bbe0e63011f: Status 404 returned error can't find the container with id 16fecff1a2e02127490fed2348591c616e48b5ce414e6285c0518bbe0e63011f
Sep 09 20:32:48.307608 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:48.307587    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcde0f8e_13e1_4cb8_bb02_f963c359b8a2.slice/crio-19b46cbd0b89eb3c9ac793880500f22686f2499c95ba5d7ce6c13d8724705f3f WatchSource:0}: Error finding container 19b46cbd0b89eb3c9ac793880500f22686f2499c95ba5d7ce6c13d8724705f3f: Status 404 returned error can't find the container with id 19b46cbd0b89eb3c9ac793880500f22686f2499c95ba5d7ce6c13d8724705f3f
Sep 09 20:32:48.327442 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.327419    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"]
Sep 09 20:32:48.331984 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:48.331963    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee110d0_1ab4_4200_8c23_65c1f5d0584d.slice/crio-18ecb7aee83f79003c5b7442798c22f2dd6bae19aca216618e1e6788361dc70c WatchSource:0}: Error finding container 18ecb7aee83f79003c5b7442798c22f2dd6bae19aca216618e1e6788361dc70c: Status 404 returned error can't find the container with id 18ecb7aee83f79003c5b7442798c22f2dd6bae19aca216618e1e6788361dc70c
Sep 09 20:32:48.345213 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.345189    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-krxjb"]
Sep 09 20:32:48.347605 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:48.347586    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f31112_a172_4677_be02_544c74a0e0aa.slice/crio-bea8ab2c3e67ed1cfe38ccd87ba986183834fa9068be0447062cfc8ec812f749 WatchSource:0}: Error finding container bea8ab2c3e67ed1cfe38ccd87ba986183834fa9068be0447062cfc8ec812f749: Status 404 returned error can't find the container with id bea8ab2c3e67ed1cfe38ccd87ba986183834fa9068be0447062cfc8ec812f749
Sep 09 20:32:48.436560 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.436528    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:48.436694 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.436628    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:48.436694 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:48.436673    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:32:48.436694 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:48.436689    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db66467bd-l6hrj: secret "image-registry-tls" not found
Sep 09 20:32:48.436833 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:48.436741    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls podName:b5b961c3-8408-4e91-ac9f-91b4c8baab66 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:49.436726768 +0000 UTC m=+29.587399090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls") pod "image-registry-db66467bd-l6hrj" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66") : secret "image-registry-tls" not found
Sep 09 20:32:48.436833 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:48.436740    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:32:48.436833 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:48.436794    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls podName:ef22136f-b71c-48bf-83ee-01d0881f8866 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:49.436777206 +0000 UTC m=+29.587449531 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls") pod "dns-default-2jfmw" (UID: "ef22136f-b71c-48bf-83ee-01d0881f8866") : secret "dns-default-metrics-tls" not found
Sep 09 20:32:48.497232 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.497198    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx" event={"ID":"af804414-5ad6-4aec-a47b-b2e47879ee82","Type":"ContainerStarted","Data":"16fecff1a2e02127490fed2348591c616e48b5ce414e6285c0518bbe0e63011f"}
Sep 09 20:32:48.498332 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.498314    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl" event={"ID":"0ee110d0-1ab4-4200-8c23-65c1f5d0584d","Type":"ContainerStarted","Data":"18ecb7aee83f79003c5b7442798c22f2dd6bae19aca216618e1e6788361dc70c"}
Sep 09 20:32:48.499513 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.499485    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" event={"ID":"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2","Type":"ContainerStarted","Data":"19b46cbd0b89eb3c9ac793880500f22686f2499c95ba5d7ce6c13d8724705f3f"}
Sep 09 20:32:48.501838 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.501819    2550 generic.go:356] "Generic (PLEG): container finished" podID="9cf69c29-0898-41f4-8484-99472336cd97" containerID="c339f5f5e0cdb5afadb8d031d6942920a9699e11714dcc69697f5b51ae556cc7" exitCode=0
Sep 09 20:32:48.501908 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.501884    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5pdg" event={"ID":"9cf69c29-0898-41f4-8484-99472336cd97","Type":"ContainerDied","Data":"c339f5f5e0cdb5afadb8d031d6942920a9699e11714dcc69697f5b51ae556cc7"}
Sep 09 20:32:48.503139 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.503118    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-krxjb" event={"ID":"38f31112-a172-4677-be02-544c74a0e0aa","Type":"ContainerStarted","Data":"bea8ab2c3e67ed1cfe38ccd87ba986183834fa9068be0447062cfc8ec812f749"}
Sep 09 20:32:48.697684 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.697593    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:48.697828 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.697771    2550 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 20:32:48.698083 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.698066    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b2m24"
Sep 09 20:32:48.758318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:48.758293    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:49.273813 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.273723    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:49.274181 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.274152    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:49.275319 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.274843    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:49.277246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.276575    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 20:32:49.277246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.276801    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 20:32:49.277246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.276824    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 20:32:49.277246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.276988    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 20:32:49.277246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.277091    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7r4f4\""
Sep 09 20:32:49.277530 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.277355    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9w8hr\""
Sep 09 20:32:49.444859 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.443656    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:49.444859 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:49.443723    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:49.444859 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:49.443929    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:32:49.444859 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:49.443945    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db66467bd-l6hrj: secret "image-registry-tls" not found
Sep 09 20:32:49.444859 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:49.444003    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls podName:b5b961c3-8408-4e91-ac9f-91b4c8baab66 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:51.443984662 +0000 UTC m=+31.594656988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls") pod "image-registry-db66467bd-l6hrj" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66") : secret "image-registry-tls" not found
Sep 09 20:32:49.444859 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:49.444740    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:32:49.444859 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:49.444787    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls podName:ef22136f-b71c-48bf-83ee-01d0881f8866 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:51.444770228 +0000 UTC m=+31.595442555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls") pod "dns-default-2jfmw" (UID: "ef22136f-b71c-48bf-83ee-01d0881f8866") : secret "dns-default-metrics-tls" not found
Sep 09 20:32:51.055414 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:51.055373    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:51.108311 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:51.108283    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:51.460903 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:51.460868    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:51.461072 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:51.460957    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:51.461072 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:51.461035    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:32:51.461193 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:51.461124    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls podName:ef22136f-b71c-48bf-83ee-01d0881f8866 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:55.461102654 +0000 UTC m=+35.611774978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls") pod "dns-default-2jfmw" (UID: "ef22136f-b71c-48bf-83ee-01d0881f8866") : secret "dns-default-metrics-tls" not found
Sep 09 20:32:51.461193 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:51.461129    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:32:51.461193 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:51.461150    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db66467bd-l6hrj: secret "image-registry-tls" not found
Sep 09 20:32:51.461357 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:51.461233    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls podName:b5b961c3-8408-4e91-ac9f-91b4c8baab66 nodeName:}" failed. No retries permitted until 2026-09-09 20:32:55.461215137 +0000 UTC m=+35.611887462 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls") pod "image-registry-db66467bd-l6hrj" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66") : secret "image-registry-tls" not found
Sep 09 20:32:52.975760 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:52.975529    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:32:52.975760 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:52.975650    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:52.975760 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:52.975709    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 20:32:52.976335 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:52.975795    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs podName:364732c6-5a16-44a2-afcf-9aaa6038c1f4 nodeName:}" failed. No retries permitted until 2026-09-09 20:33:24.975772157 +0000 UTC m=+65.126444478 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs") pod "network-metrics-daemon-jnrt5" (UID: "364732c6-5a16-44a2-afcf-9aaa6038c1f4") : secret "metrics-daemon-secret" not found
Sep 09 20:32:52.979490 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:52.979346    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca0688d-9168-4fe9-8ee9-4db243c38e7d-original-pull-secret\") pod \"global-pull-secret-syncer-z64xx\" (UID: \"eca0688d-9168-4fe9-8ee9-4db243c38e7d\") " pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:53.076616 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:53.076583    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7r5d\" (UniqueName: \"kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d\") pod \"network-check-target-xrfrn\" (UID: \"f28e9631-b9b8-4f66-b180-04b10282b574\") " pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:53.079476 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:53.079447    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7r5d\" (UniqueName: \"kubernetes.io/projected/f28e9631-b9b8-4f66-b180-04b10282b574-kube-api-access-l7r5d\") pod \"network-check-target-xrfrn\" (UID: \"f28e9631-b9b8-4f66-b180-04b10282b574\") " pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:53.176480 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:53.176445    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:53.202577 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:53.202553    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:32:53.208593 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:53.208568    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z64xx"
Sep 09 20:32:53.238899 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:53.238872    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:54.254178 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:54.254131    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:54.308018 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:54.307986    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:55.498357 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:55.498316    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:32:55.498925 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:55.498403    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:32:55.498925 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:55.498453    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:32:55.498925 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:55.498528    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls podName:ef22136f-b71c-48bf-83ee-01d0881f8866 nodeName:}" failed. No retries permitted until 2026-09-09 20:33:03.498506394 +0000 UTC m=+43.649178719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls") pod "dns-default-2jfmw" (UID: "ef22136f-b71c-48bf-83ee-01d0881f8866") : secret "dns-default-metrics-tls" not found
Sep 09 20:32:55.498925 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:55.498557    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:32:55.498925 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:55.498571    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db66467bd-l6hrj: secret "image-registry-tls" not found
Sep 09 20:32:55.498925 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:32:55.498622    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls podName:b5b961c3-8408-4e91-ac9f-91b4c8baab66 nodeName:}" failed. No retries permitted until 2026-09-09 20:33:03.498605465 +0000 UTC m=+43.649277788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls") pod "image-registry-db66467bd-l6hrj" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66") : secret "image-registry-tls" not found
Sep 09 20:32:56.126606 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:56.126574    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:56.227281 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:56.227248    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:56.328367 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:56.328329    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:57.145425 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:57.145377    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:57.204061 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:57.204030    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:58.226321 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:58.226286    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:58.286549 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:58.286516    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:59.810543 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:59.810469    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:59.854487 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:59.854463    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xrfrn"]
Sep 09 20:32:59.857635 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:59.857603    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf28e9631_b9b8_4f66_b180_04b10282b574.slice/crio-876d4f80fd3ae1f3db53bb644d1ddbcba4b2da007da33c59ba6474120b7704c5 WatchSource:0}: Error finding container 876d4f80fd3ae1f3db53bb644d1ddbcba4b2da007da33c59ba6474120b7704c5: Status 404 returned error can't find the container with id 876d4f80fd3ae1f3db53bb644d1ddbcba4b2da007da33c59ba6474120b7704c5
Sep 09 20:32:59.859946 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:59.859922    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z64xx"]
Sep 09 20:32:59.863819 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:32:59.863788    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca0688d_9168_4fe9_8ee9_4db243c38e7d.slice/crio-c7a2e3823db5a94b70612d113dcb756317b1edc19bf6c87a4daad2236d31d0ac WatchSource:0}: Error finding container c7a2e3823db5a94b70612d113dcb756317b1edc19bf6c87a4daad2236d31d0ac: Status 404 returned error can't find the container with id c7a2e3823db5a94b70612d113dcb756317b1edc19bf6c87a4daad2236d31d0ac
Sep 09 20:32:59.912029 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:32:59.911999    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:00.540214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.540148    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xrfrn" event={"ID":"f28e9631-b9b8-4f66-b180-04b10282b574","Type":"ContainerStarted","Data":"876d4f80fd3ae1f3db53bb644d1ddbcba4b2da007da33c59ba6474120b7704c5"}
Sep 09 20:33:00.541394 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.541366    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z64xx" event={"ID":"eca0688d-9168-4fe9-8ee9-4db243c38e7d","Type":"ContainerStarted","Data":"c7a2e3823db5a94b70612d113dcb756317b1edc19bf6c87a4daad2236d31d0ac"}
Sep 09 20:33:00.543033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.543013    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx" event={"ID":"af804414-5ad6-4aec-a47b-b2e47879ee82","Type":"ContainerStarted","Data":"b06a1811a697065673b8b6dfbb33a468335fd4ab53269dd292adc6133b5be02b"}
Sep 09 20:33:00.544682 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.544655    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl" event={"ID":"0ee110d0-1ab4-4200-8c23-65c1f5d0584d","Type":"ContainerStarted","Data":"d162439fbbd473666a4dda1992d182b7b8ff4a1949c34a60572b57424053a261"}
Sep 09 20:33:00.544804 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.544778    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:33:00.546927 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.546896    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl"
Sep 09 20:33:00.547220 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.547196    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" event={"ID":"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2","Type":"ContainerStarted","Data":"aea17081fc7be981b4a6302f49b68cc51118d1a2434e327b3d37aa72663c47c9"}
Sep 09 20:33:00.547312 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.547234    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" event={"ID":"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2","Type":"ContainerStarted","Data":"0420bd95e0d3dc55e551cfb7704f4c51b60220243f69fb888d9f2817863f03b1"}
Sep 09 20:33:00.547312 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.547252    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" event={"ID":"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2","Type":"ContainerStarted","Data":"dbd88ab7f6a8e40e312033e69c1aef8d8b3582c28314337c4251eaa7ef66e840"}
Sep 09 20:33:00.550152 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.550131    2550 generic.go:356] "Generic (PLEG): container finished" podID="9cf69c29-0898-41f4-8484-99472336cd97" containerID="c9616a094667e4871ab4db38092309d94f8c56b632e05c26137e6e5213ce2fd9" exitCode=0
Sep 09 20:33:00.550271 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.550196    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5pdg" event={"ID":"9cf69c29-0898-41f4-8484-99472336cd97","Type":"ContainerDied","Data":"c9616a094667e4871ab4db38092309d94f8c56b632e05c26137e6e5213ce2fd9"}
Sep 09 20:33:00.551741 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.551721    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-krxjb" event={"ID":"38f31112-a172-4677-be02-544c74a0e0aa","Type":"ContainerStarted","Data":"14270f70807ba2b20ba6b8ee37c45e01b3e7ed6b0ee766ec8d866142b8870018"}
Sep 09 20:33:00.558581 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.558541    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb65c7858-gdtjx" podStartSLOduration=28.28687602 podStartE2EDuration="39.558528189s" totalImagesPullingTime="11.271652169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:00.557700144 +0000 UTC m=+40.708372492" watchObservedRunningTime="2026-09-09 20:33:00.558528189 +0000 UTC m=+40.709200531"
Sep 09 20:33:00.573953 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.573902    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" podStartSLOduration=28.301284396 podStartE2EDuration="39.573887023s" totalImagesPullingTime="11.272602627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:00.573361113 +0000 UTC m=+40.724033477" watchObservedRunningTime="2026-09-09 20:33:00.573887023 +0000 UTC m=+40.724559367"
Sep 09 20:33:00.591340 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.590967    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c589674c-qxtwl" podStartSLOduration=28.338913526 podStartE2EDuration="39.59094759s" totalImagesPullingTime="11.252034064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:00.590722513 +0000 UTC m=+40.741394857" watchObservedRunningTime="2026-09-09 20:33:00.59094759 +0000 UTC m=+40.741619934"
Sep 09 20:33:00.605230 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.605188    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-krxjb" podStartSLOduration=2.373470848 podStartE2EDuration="13.605147809s" totalImagesPullingTime="11.231676961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:32:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:00.604227097 +0000 UTC m=+40.754899440" watchObservedRunningTime="2026-09-09 20:33:00.605147809 +0000 UTC m=+40.755820154"
Sep 09 20:33:00.767878 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.767847    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:00.867728 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:00.867632    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:01.275300 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:01.275259    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:01.559608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:01.559520    2550 generic.go:356] "Generic (PLEG): container finished" podID="9cf69c29-0898-41f4-8484-99472336cd97" containerID="7f9899d2351a71034d0bdb7684cbae6cffafef941e470449864067d1034a9fbe" exitCode=0
Sep 09 20:33:01.559608 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:01.559559    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5pdg" event={"ID":"9cf69c29-0898-41f4-8484-99472336cd97","Type":"ContainerDied","Data":"7f9899d2351a71034d0bdb7684cbae6cffafef941e470449864067d1034a9fbe"}
Sep 09 20:33:02.570496 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:02.570409    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5pdg" event={"ID":"9cf69c29-0898-41f4-8484-99472336cd97","Type":"ContainerStarted","Data":"910a31399924ddbbd9d845d186b9e6cdc64d1c7f337ffc2cb65c3aa97c6d7683"}
Sep 09 20:33:02.594286 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:02.594230    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z5pdg" podStartSLOduration=11.627774553 podStartE2EDuration="42.594214608s" totalImagesPullingTime="30.966440055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:02.593702887 +0000 UTC m=+42.744375243" watchObservedRunningTime="2026-09-09 20:33:02.594214608 +0000 UTC m=+42.744886950"
Sep 09 20:33:02.720496 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:02.720465    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:02.780856 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:02.780815    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:03.564575 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:03.564535    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:33:03.564763 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:03.564626    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:33:03.564763 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:03.564679    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:33:03.564763 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:03.564699    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db66467bd-l6hrj: secret "image-registry-tls" not found
Sep 09 20:33:03.564763 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:03.564730    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:33:03.564961 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:03.564772    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls podName:b5b961c3-8408-4e91-ac9f-91b4c8baab66 nodeName:}" failed. No retries permitted until 2026-09-09 20:33:19.564750724 +0000 UTC m=+59.715423058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls") pod "image-registry-db66467bd-l6hrj" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66") : secret "image-registry-tls" not found
Sep 09 20:33:03.564961 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:03.564792    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls podName:ef22136f-b71c-48bf-83ee-01d0881f8866 nodeName:}" failed. No retries permitted until 2026-09-09 20:33:19.564783186 +0000 UTC m=+59.715455511 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls") pod "dns-default-2jfmw" (UID: "ef22136f-b71c-48bf-83ee-01d0881f8866") : secret "dns-default-metrics-tls" not found
Sep 09 20:33:05.581181 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:05.581133    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xrfrn" event={"ID":"f28e9631-b9b8-4f66-b180-04b10282b574","Type":"ContainerStarted","Data":"f2165e21cfc6c39dbe9b17147ddcc153f8dc3f7ee72740fb67dfa7235b453a25"}
Sep 09 20:33:05.581617 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:05.581235    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:33:05.582553 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:05.582530    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z64xx" event={"ID":"eca0688d-9168-4fe9-8ee9-4db243c38e7d","Type":"ContainerStarted","Data":"e3c1f00fd99795becbd8e0fbc44f9880ee20eb1d4c8967e1e7fd3b8b2291613e"}
Sep 09 20:33:05.601092 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:05.601054    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xrfrn" podStartSLOduration=40.550538712 podStartE2EDuration="45.601041577s" totalImagesPullingTime="5.050502865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:05.599892277 +0000 UTC m=+45.750564620" watchObservedRunningTime="2026-09-09 20:33:05.601041577 +0000 UTC m=+45.751713950"
Sep 09 20:33:05.618195 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:05.618144    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z64xx" podStartSLOduration=40.560773087 podStartE2EDuration="45.618133097s" totalImagesPullingTime="5.05736001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:05.617416703 +0000 UTC m=+45.768089052" watchObservedRunningTime="2026-09-09 20:33:05.618133097 +0000 UTC m=+45.768805439"
Sep 09 20:33:05.924462 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:05.924391    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:06.030836 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.030809    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:06.789676 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.789645    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5ltp2"]
Sep 09 20:33:06.811402 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.811377    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5ltp2"]
Sep 09 20:33:06.811558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.811483    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:06.814214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.814193    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 20:33:06.814501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.814480    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 20:33:06.814581 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.814570    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ng47s\""
Sep 09 20:33:06.814843 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.814824    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 20:33:06.814947 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.814879    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 20:33:06.815498 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.815478    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:06.992472 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.992437    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbqfn\" (UniqueName: \"kubernetes.io/projected/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-api-access-mbqfn\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:06.992640 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.992479    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-data-volume\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:06.992640 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.992522    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:06.992640 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.992567    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-crio-socket\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:06.992640 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:06.992593    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.045261 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.045184    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:07.093283 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.093258    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbqfn\" (UniqueName: \"kubernetes.io/projected/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-api-access-mbqfn\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.093405 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.093291    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-data-volume\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.093405 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.093321    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.093405 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.093350    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-crio-socket\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.093405 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:07.093390    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:33:07.093723 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:07.093461    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:07.593445783 +0000 UTC m=+47.744118103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : configmap "kube-rbac-proxy" not found
Sep 09 20:33:07.093723 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.093485    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-crio-socket\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.093723 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.093494    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.093723 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:07.093573    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:07.093723 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:07.093630    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:07.593618052 +0000 UTC m=+47.744290373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:07.093892 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.093731    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-data-volume\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.105477 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.105456    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbqfn\" (UniqueName: \"kubernetes.io/projected/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-api-access-mbqfn\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.157979 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.157957    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:07.225790 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.225766    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:07.433992 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.433905    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:07.542452 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.542422    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:07.597932 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.597901    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.598044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.597945    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:07.598044 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:07.598022    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:33:07.598122 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:07.598066    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:07.598122 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:07.598095    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:08.598075465 +0000 UTC m=+48.748747806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : configmap "kube-rbac-proxy" not found
Sep 09 20:33:07.598122 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:07.598118    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:08.598103491 +0000 UTC m=+48.748775816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:07.726805 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.726775    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:07.875269 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:07.875238    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:08.131926 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:08.131851    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b5k8p_8f54dbf2-1cc5-4c0e-91c0-8dc53551965e/dns-node-resolver/0.log"
Sep 09 20:33:08.195798 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:08.195758    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:08.304354 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:08.304327    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:08.604623 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:08.604595    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:08.604755 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:08.604641    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:08.604755 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:08.604718    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:33:08.604837 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:08.604780    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:10.604763218 +0000 UTC m=+50.755435540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : configmap "kube-rbac-proxy" not found
Sep 09 20:33:08.604837 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:08.604784    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:08.604837 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:08.604819    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:10.604809079 +0000 UTC m=+50.755481409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:09.125534 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:09.125507    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k6x8t_fbfc5682-d88a-45cf-b4c2-aac06e054300/node-ca/0.log"
Sep 09 20:33:09.277001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:09.276969    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:09.340391 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:09.340367    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:09.853810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:09.853769    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:09.924383 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:09.924354    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-krxjb_38f31112-a172-4677-be02-544c74a0e0aa/serve-healthcheck-canary/0.log"
Sep 09 20:33:09.951154 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:09.951125    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:10.617806 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:10.617776    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:10.618187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:10.617821    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:10.618187 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:10.617912    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:33:10.618187 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:10.617939    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:10.618187 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:10.617978    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:14.617959759 +0000 UTC m=+54.768632093 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : configmap "kube-rbac-proxy" not found
Sep 09 20:33:10.618187 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:10.617991    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:14.617985804 +0000 UTC m=+54.768658125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:10.894013 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:10.893929    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:13.334224 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:13.334190    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:13.435837 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:13.435808    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:14.641524 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:14.641477    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:14.641524 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:14.641536    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:14.641941 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:14.641599    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:33:14.641941 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:14.641652    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:14.641941 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:14.641658    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:22.641642402 +0000 UTC m=+62.792314724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : configmap "kube-rbac-proxy" not found
Sep 09 20:33:14.641941 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:14.641702    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:22.641688712 +0000 UTC m=+62.792361032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:15.142557 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:15.142527    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:15.532295 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:15.532260    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:15.712272 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:15.712241    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:15.809944 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:15.809864    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:15.920904 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:15.920872    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:16.048783 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:16.048751    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:16.144445 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:16.144368    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:16.242038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:16.242010    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:16.800229 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:16.800196    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:17.206373 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:17.206296    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:17.987550 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:17.987519    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:18.389382 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:18.389298    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:18.590431 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:18.590400    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:18.815195 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:18.815146    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:19.198283 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.198203    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:19.387901 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.387872    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:19.576589 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.576557    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:33:19.576697 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.576616    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:33:19.578824 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.578792    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef22136f-b71c-48bf-83ee-01d0881f8866-metrics-tls\") pod \"dns-default-2jfmw\" (UID: \"ef22136f-b71c-48bf-83ee-01d0881f8866\") " pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:33:19.578938 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.578871    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") pod \"image-registry-db66467bd-l6hrj\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") " pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:33:19.593194 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.593160    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:19.789006 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.788979    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:19.791424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.791408    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:33:19.841187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.841108    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:33:19.903263 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:19.903209    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:20.074022 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.073988    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db66467bd-l6hrj"]
Sep 09 20:33:20.078230 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:33:20.078199    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b961c3_8408_4e91_ac9f_91b4c8baab66.slice/crio-54afacc85be930b01bddbe19c819aced4ec292e5d54953b0289dc17eb983d1d4 WatchSource:0}: Error finding container 54afacc85be930b01bddbe19c819aced4ec292e5d54953b0289dc17eb983d1d4: Status 404 returned error can't find the container with id 54afacc85be930b01bddbe19c819aced4ec292e5d54953b0289dc17eb983d1d4
Sep 09 20:33:20.127236 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.127211    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2jfmw"]
Sep 09 20:33:20.138072 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:33:20.138047    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef22136f_b71c_48bf_83ee_01d0881f8866.slice/crio-dabf080233b2b5fd43a009c78e887f1f4e3723ba6afe66334c84b4292f31d7e5 WatchSource:0}: Error finding container dabf080233b2b5fd43a009c78e887f1f4e3723ba6afe66334c84b4292f31d7e5: Status 404 returned error can't find the container with id dabf080233b2b5fd43a009c78e887f1f4e3723ba6afe66334c84b4292f31d7e5
Sep 09 20:33:20.387652 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.387577    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:20.448788 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.448762    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:20.590449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.590416    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:20.630536 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.630502    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2jfmw" event={"ID":"ef22136f-b71c-48bf-83ee-01d0881f8866","Type":"ContainerStarted","Data":"dabf080233b2b5fd43a009c78e887f1f4e3723ba6afe66334c84b4292f31d7e5"}
Sep 09 20:33:20.631974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.631942    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db66467bd-l6hrj" event={"ID":"b5b961c3-8408-4e91-ac9f-91b4c8baab66","Type":"ContainerStarted","Data":"27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3"}
Sep 09 20:33:20.631974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.631975    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db66467bd-l6hrj" event={"ID":"b5b961c3-8408-4e91-ac9f-91b4c8baab66","Type":"ContainerStarted","Data":"54afacc85be930b01bddbe19c819aced4ec292e5d54953b0289dc17eb983d1d4"}
Sep 09 20:33:20.632137 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.632086    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:33:20.650952 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.650877    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-db66467bd-l6hrj" podStartSLOduration=60.650863045 podStartE2EDuration="1m0.650863045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:20.650080519 +0000 UTC m=+60.800752863" watchObservedRunningTime="2026-09-09 20:33:20.650863045 +0000 UTC m=+60.801535401"
Sep 09 20:33:20.798848 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:20.798821    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:21.058763 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:21.058732    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:21.113739 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:21.113711    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:21.183552 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:21.183518    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:21.780427 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:21.780396    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:22.003016 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:22.002990    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:22.189383 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:22.189316    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:22.592913 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:22.592881    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:22.698943 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:22.698907    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:22.699122 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:22.698959    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:22.699122 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:22.699117    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:22.699236 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:33:22.699190    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls podName:9c30feed-f1d0-44a7-9764-e0a7f4183d6e nodeName:}" failed. No retries permitted until 2026-09-09 20:33:38.699158987 +0000 UTC m=+78.849831325 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5ltp2" (UID: "9c30feed-f1d0-44a7-9764-e0a7f4183d6e") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:33:22.699503 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:22.699482    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:22.734449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:22.734396    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:22.835256 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:22.835220    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:23.189185 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:23.189101    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:23.644431 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:23.644397    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2jfmw" event={"ID":"ef22136f-b71c-48bf-83ee-01d0881f8866","Type":"ContainerStarted","Data":"ef5ac938ee31d9d0836cb5fccf73f9193d4a1f4e68eefae44050a73a9a707654"}
Sep 09 20:33:23.644589 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:23.644445    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2jfmw" event={"ID":"ef22136f-b71c-48bf-83ee-01d0881f8866","Type":"ContainerStarted","Data":"af894b6813280ba5ba7c58bea84975bc006ac3339c661f213b69a77004071fb7"}
Sep 09 20:33:23.644589 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:23.644526    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:33:23.664082 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:23.664042    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2jfmw" podStartSLOduration=33.966594364 podStartE2EDuration="36.664029222s" totalImagesPullingTime="2.697434858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:32:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:23.663066982 +0000 UTC m=+63.813739336" watchObservedRunningTime="2026-09-09 20:33:23.664029222 +0000 UTC m=+63.814701564"
Sep 09 20:33:23.791804 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:23.791776    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:24.390947 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:24.390911    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:24.594400 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:24.594369    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:24.809967 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:24.809936    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:25.015839 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:25.015788    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:33:25.018058 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:25.018027    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364732c6-5a16-44a2-afcf-9aaa6038c1f4-metrics-certs\") pod \"network-metrics-daemon-jnrt5\" (UID: \"364732c6-5a16-44a2-afcf-9aaa6038c1f4\") " pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:33:25.187138 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:25.187053    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:25.292955 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:25.292924    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9w8hr\""
Sep 09 20:33:25.302311 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:25.302280    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnrt5"
Sep 09 20:33:25.566255 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:25.566225    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jnrt5"]
Sep 09 20:33:25.568959 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:33:25.568937    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364732c6_5a16_44a2_afcf_9aaa6038c1f4.slice/crio-d05200164109bb89d7d5b5ecbe46216e9d307216067789acb6d35447ff7168b8 WatchSource:0}: Error finding container d05200164109bb89d7d5b5ecbe46216e9d307216067789acb6d35447ff7168b8: Status 404 returned error can't find the container with id d05200164109bb89d7d5b5ecbe46216e9d307216067789acb6d35447ff7168b8
Sep 09 20:33:25.650706 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:25.650675    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jnrt5" event={"ID":"364732c6-5a16-44a2-afcf-9aaa6038c1f4","Type":"ContainerStarted","Data":"d05200164109bb89d7d5b5ecbe46216e9d307216067789acb6d35447ff7168b8"}
Sep 09 20:33:25.786265 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:25.786234    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:25.983552 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:25.983522    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:26.183568 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:26.183501    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:26.656146 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:26.656053    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jnrt5" event={"ID":"364732c6-5a16-44a2-afcf-9aaa6038c1f4","Type":"ContainerStarted","Data":"addddc8e69323c5f00e489cf6217c87819f56023f49c97a064599a07bb78f508"}
Sep 09 20:33:26.656146 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:26.656107    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jnrt5" event={"ID":"364732c6-5a16-44a2-afcf-9aaa6038c1f4","Type":"ContainerStarted","Data":"9445ea6c657b234f36eef0a7bb8c61c14cf41229d55891b4339d2e4be8ba8695"}
Sep 09 20:33:26.672892 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:26.672844    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jnrt5" podStartSLOduration=65.92443834 podStartE2EDuration="1m6.672829135s" totalImagesPullingTime="748.390795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:26.67142933 +0000 UTC m=+66.822101673" watchObservedRunningTime="2026-09-09 20:33:26.672829135 +0000 UTC m=+66.823501504"
Sep 09 20:33:30.196654 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:30.196620    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:30.347021 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:30.346993    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:30.425805 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:30.425768    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:33.248309 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:33.248281    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:33.348071 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:33.348045    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:33.423292 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:33.423267    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:33.649202 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:33.649121    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2jfmw"
Sep 09 20:33:33.720952 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:33.720902    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:33.965703 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:33.965666    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:34.035895 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:34.035852    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:35.391961 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:35.391931    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:35.446930 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:35.446903    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:36.430956 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:36.430923    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:36.506918 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:36.506889    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:36.588424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:36.588400    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xrfrn"
Sep 09 20:33:36.689911 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:36.689839    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:38.718147 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:38.718116    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:38.720463 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:38.720442    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c30feed-f1d0-44a7-9764-e0a7f4183d6e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5ltp2\" (UID: \"9c30feed-f1d0-44a7-9764-e0a7f4183d6e\") " pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:38.923082 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:38.923056    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ng47s\""
Sep 09 20:33:38.931694 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:38.931677    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5ltp2"
Sep 09 20:33:39.236890 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:39.236810    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5ltp2"]
Sep 09 20:33:39.239692 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:33:39.239665    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c30feed_f1d0_44a7_9764_e0a7f4183d6e.slice/crio-b123bdf13ec05750c6d439bbfda82aa4bad533e92013881bc8033d2d935f40bb WatchSource:0}: Error finding container b123bdf13ec05750c6d439bbfda82aa4bad533e92013881bc8033d2d935f40bb: Status 404 returned error can't find the container with id b123bdf13ec05750c6d439bbfda82aa4bad533e92013881bc8033d2d935f40bb
Sep 09 20:33:39.700497 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:39.700413    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5ltp2" event={"ID":"9c30feed-f1d0-44a7-9764-e0a7f4183d6e","Type":"ContainerStarted","Data":"a47a6a52b884cb82c03ce8e707276826f0b1e4d43851a90fe63424844b75fb19"}
Sep 09 20:33:39.700497 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:39.700466    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5ltp2" event={"ID":"9c30feed-f1d0-44a7-9764-e0a7f4183d6e","Type":"ContainerStarted","Data":"b123bdf13ec05750c6d439bbfda82aa4bad533e92013881bc8033d2d935f40bb"}
Sep 09 20:33:39.773869 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:39.773813    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:39.795927 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:39.795896    2550 patch_prober.go:28] interesting pod/image-registry-db66467bd-l6hrj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 09 20:33:39.796071 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:39.795948    2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-db66467bd-l6hrj" podUID="b5b961c3-8408-4e91-ac9f-91b4c8baab66" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 20:33:40.705607 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:40.705572    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5ltp2" event={"ID":"9c30feed-f1d0-44a7-9764-e0a7f4183d6e","Type":"ContainerStarted","Data":"f7e482a292dfa5f69e2d200f8e0ecf0c5652d16cfa7220f8d692432d1e7f1d4f"}
Sep 09 20:33:41.601488 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.601449    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:41.640036 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.640012    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:33:41.808084 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.808016    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:41.863563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.863493    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-76nqd"]
Sep 09 20:33:41.866109 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.866082    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:41.867893 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.867868    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 20:33:41.868297 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.868277    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 20:33:41.868297 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.868295    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 20:33:41.868490 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.868329    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-97pg4\""
Sep 09 20:33:41.868490 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.868298    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 20:33:41.868490 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.868399    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 20:33:41.868749 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.868733    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 20:33:41.894016 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.893989    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:41.938763 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.938733    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-textfile\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:41.938915 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.938775    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr95m\" (UniqueName: \"kubernetes.io/projected/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-kube-api-access-sr95m\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:41.938915 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.938803    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-tls\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:41.938915 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.938900    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-wtmp\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:41.939040 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.938945    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-sys\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:41.939040 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.939004    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-metrics-client-ca\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:41.939121 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.939061    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:41.939121 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.939096    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-accelerators-collector-config\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:41.939245 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.939219    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-root\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:41.982571 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:41.982533    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:42.040355 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040320    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-textfile\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040355 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040363    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr95m\" (UniqueName: \"kubernetes.io/projected/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-kube-api-access-sr95m\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040579 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040469    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-tls\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040579 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040498    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-wtmp\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040579 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040526    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-sys\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040579 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040561    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-metrics-client-ca\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040593    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040622    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-accelerators-collector-config\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040681    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-root\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040723    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-sys\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040916 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040733    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-wtmp\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040916 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040752    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-textfile\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.040916 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.040759    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-root\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.041239 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.041218    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-metrics-client-ca\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.041370 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.041243    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-accelerators-collector-config\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.043024 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.043005    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.043379 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.043363    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-node-exporter-tls\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.049285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.049260    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr95m\" (UniqueName: \"kubernetes.io/projected/52aa8ec2-fa0c-47a8-be52-2fc776bc2983-kube-api-access-sr95m\") pod \"node-exporter-76nqd\" (UID: \"52aa8ec2-fa0c-47a8-be52-2fc776bc2983\") " pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.092147 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.092110    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:42.175754 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.175723    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-76nqd"
Sep 09 20:33:42.190240 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:33:42.190214    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52aa8ec2_fa0c_47a8_be52_2fc776bc2983.slice/crio-fe7f3406d93c49f9bf1c572624df8b44e57ac46b8dc8194bee5baf456462924d WatchSource:0}: Error finding container fe7f3406d93c49f9bf1c572624df8b44e57ac46b8dc8194bee5baf456462924d: Status 404 returned error can't find the container with id fe7f3406d93c49f9bf1c572624df8b44e57ac46b8dc8194bee5baf456462924d
Sep 09 20:33:42.192424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.192395    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:42.328662 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.328628    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:42.428810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.428740    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:42.706021 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.705933    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:42.714405 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.714378    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5ltp2" event={"ID":"9c30feed-f1d0-44a7-9764-e0a7f4183d6e","Type":"ContainerStarted","Data":"8db99b5663112f7981c9d162ec909d1c057bc30d7264baaede104ea1ab1c12ba"}
Sep 09 20:33:42.715586 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.715564    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76nqd" event={"ID":"52aa8ec2-fa0c-47a8-be52-2fc776bc2983","Type":"ContainerStarted","Data":"fe7f3406d93c49f9bf1c572624df8b44e57ac46b8dc8194bee5baf456462924d"}
Sep 09 20:33:42.736105 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.736053    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5ltp2" podStartSLOduration=34.027195283 podStartE2EDuration="36.736036004s" totalImagesPullingTime="2.708840721s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:33:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:42.735210078 +0000 UTC m=+82.885882421" watchObservedRunningTime="2026-09-09 20:33:42.736036004 +0000 UTC m=+82.886708395"
Sep 09 20:33:42.804713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.804681    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:42.861818 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.861690    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:42.930859 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.930823    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:42.997195 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:42.997151    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:43.720213 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:43.720182    2550 generic.go:356] "Generic (PLEG): container finished" podID="52aa8ec2-fa0c-47a8-be52-2fc776bc2983" containerID="4550b34eeacd0fefe3f7f7e1b38685e3c0390abb915c5bf1d262c016458f425d" exitCode=0
Sep 09 20:33:43.720600 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:43.720270    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76nqd" event={"ID":"52aa8ec2-fa0c-47a8-be52-2fc776bc2983","Type":"ContainerDied","Data":"4550b34eeacd0fefe3f7f7e1b38685e3c0390abb915c5bf1d262c016458f425d"}
Sep 09 20:33:44.725450 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:44.725412    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76nqd" event={"ID":"52aa8ec2-fa0c-47a8-be52-2fc776bc2983","Type":"ContainerStarted","Data":"f6ce2f7581952c20d5eb6e062b53b343542054df94193b4ab8425c1e5b1c5f40"}
Sep 09 20:33:44.725911 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:44.725466    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76nqd" event={"ID":"52aa8ec2-fa0c-47a8-be52-2fc776bc2983","Type":"ContainerStarted","Data":"7bbe37201283341813859a0a67832f6f313ca93c689dd7c49848ea83df028b08"}
Sep 09 20:33:44.748607 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:44.748560    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-76nqd" podStartSLOduration=2.61513735 podStartE2EDuration="3.748543829s" totalImagesPullingTime="1.072080946s" totalInitContainerRuntime="61.325533ms" isStatefulPod=true podCreationTimestamp="2026-09-09 20:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:44.746964819 +0000 UTC m=+84.897637199" watchObservedRunningTime="2026-09-09 20:33:44.748543829 +0000 UTC m=+84.899216172"
Sep 09 20:33:44.778363 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:44.778326    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:44.841122 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:44.841089    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:45.026813 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:45.026727    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:45.283663 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:45.283585    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:46.604684 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:46.604654    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:46.771185 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:46.771144    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:48.065852 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:48.065817    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:51.063026 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:51.062993    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:51.118515 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:51.118484    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:51.776447 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:51.776412    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:51.829434 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:51.829406    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:52.195075 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:52.195005    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-db66467bd-l6hrj"]
Sep 09 20:33:52.266012 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:52.265984    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:52.320785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:52.320753    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:52.418577 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:52.418552    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:52.520119 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:52.520093    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:54.126486 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:54.126453    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:54.226309 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:54.226277    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:54.279513 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:54.279487    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:54.409675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:54.409601    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:54.589462 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:54.589430    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:54.992075 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:54.992040    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:55.052760 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:55.052728    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:55.195002 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:55.194954    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:55.261072 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:55.261006    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:55.813587 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:55.813549    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:56.042489 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:56.042452    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:56.136893 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:56.136806    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:56.237350 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:56.237318    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:56.337456 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:56.337429    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:56.391357 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:33:56.391297    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:01.213033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:01.213001    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:01.315177 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:01.315147    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:03.106985 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:03.106950    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:05.383181 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:05.383131    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:05.480274 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:05.480248    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:05.878575 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:05.878540    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:05.938944 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:05.938910    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:07.242179 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:07.242130    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:07.342856 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:07.342820    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:11.818800 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:11.818770    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-krxjb_38f31112-a172-4677-be02-544c74a0e0aa/serve-healthcheck-canary/0.log"
Sep 09 20:34:17.213975 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.213907    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-db66467bd-l6hrj" podUID="b5b961c3-8408-4e91-ac9f-91b4c8baab66" containerName="registry" containerID="cri-o://27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3" gracePeriod=30
Sep 09 20:34:17.449842 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.449821    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:34:17.584087 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.584015    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b5b961c3-8408-4e91-ac9f-91b4c8baab66-ca-trust-extracted\") pod \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") "
Sep 09 20:34:17.584087 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.584046    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-trusted-ca\") pod \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") "
Sep 09 20:34:17.584087 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.584082    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-image-registry-private-configuration\") pod \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") "
Sep 09 20:34:17.584336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.584219    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57hpr\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-kube-api-access-57hpr\") pod \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") "
Sep 09 20:34:17.584336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.584266    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-installation-pull-secrets\") pod \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") "
Sep 09 20:34:17.584336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.584290    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") pod \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") "
Sep 09 20:34:17.584336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.584324    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-bound-sa-token\") pod \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") "
Sep 09 20:34:17.584531 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.584365    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-certificates\") pod \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\" (UID: \"b5b961c3-8408-4e91-ac9f-91b4c8baab66\") "
Sep 09 20:34:17.584531 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.584502    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b5b961c3-8408-4e91-ac9f-91b4c8baab66" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:34:17.584900 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.584865    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b5b961c3-8408-4e91-ac9f-91b4c8baab66" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:34:17.586797 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.586771    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b5b961c3-8408-4e91-ac9f-91b4c8baab66" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:34:17.586899 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.586855    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-kube-api-access-57hpr" (OuterVolumeSpecName: "kube-api-access-57hpr") pod "b5b961c3-8408-4e91-ac9f-91b4c8baab66" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66"). InnerVolumeSpecName "kube-api-access-57hpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:34:17.586899 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.586873    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b5b961c3-8408-4e91-ac9f-91b4c8baab66" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:34:17.586989 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.586905    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b5b961c3-8408-4e91-ac9f-91b4c8baab66" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:34:17.586989 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.586924    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b5b961c3-8408-4e91-ac9f-91b4c8baab66" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:34:17.592563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.592540    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5b961c3-8408-4e91-ac9f-91b4c8baab66-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b5b961c3-8408-4e91-ac9f-91b4c8baab66" (UID: "b5b961c3-8408-4e91-ac9f-91b4c8baab66"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:34:17.685373 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.685327    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57hpr\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-kube-api-access-57hpr\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:34:17.685373 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.685368    2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-installation-pull-secrets\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:34:17.685373 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.685379    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-tls\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:34:17.685373 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.685388    2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5b961c3-8408-4e91-ac9f-91b4c8baab66-bound-sa-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:34:17.685615 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.685396    2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-registry-certificates\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:34:17.685615 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.685404    2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b5b961c3-8408-4e91-ac9f-91b4c8baab66-ca-trust-extracted\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:34:17.685615 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.685412    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b5b961c3-8408-4e91-ac9f-91b4c8baab66-trusted-ca\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:34:17.685615 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.685421    2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b5b961c3-8408-4e91-ac9f-91b4c8baab66-image-registry-private-configuration\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:34:17.831918 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.831881    2550 generic.go:356] "Generic (PLEG): container finished" podID="b5b961c3-8408-4e91-ac9f-91b4c8baab66" containerID="27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3" exitCode=0
Sep 09 20:34:17.832045 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.831947    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db66467bd-l6hrj"
Sep 09 20:34:17.832045 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.831959    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db66467bd-l6hrj" event={"ID":"b5b961c3-8408-4e91-ac9f-91b4c8baab66","Type":"ContainerDied","Data":"27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3"}
Sep 09 20:34:17.832045 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.832036    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db66467bd-l6hrj" event={"ID":"b5b961c3-8408-4e91-ac9f-91b4c8baab66","Type":"ContainerDied","Data":"54afacc85be930b01bddbe19c819aced4ec292e5d54953b0289dc17eb983d1d4"}
Sep 09 20:34:17.832154 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.832074    2550 scope.go:117] "RemoveContainer" containerID="27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3"
Sep 09 20:34:17.840003 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.839983    2550 scope.go:117] "RemoveContainer" containerID="27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3"
Sep 09 20:34:17.840268 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:34:17.840249    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3\": container with ID starting with 27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3 not found: ID does not exist" containerID="27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3"
Sep 09 20:34:17.840318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.840276    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3"} err="failed to get container status \"27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3\": rpc error: code = NotFound desc = could not find container \"27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3\": container with ID starting with 27497c5c4a510bf7fd9f3f59693566803f7750b0848f722572b2f6e9d54c61d3 not found: ID does not exist"
Sep 09 20:34:17.851207 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.851186    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-db66467bd-l6hrj"]
Sep 09 20:34:17.854774 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:17.854754    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-db66467bd-l6hrj"]
Sep 09 20:34:18.014047 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:18.014010    2550 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" podUID="bcde0f8e-13e1-4cb8-bb02-f963c359b8a2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 09 20:34:18.277723 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:18.277691    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5b961c3-8408-4e91-ac9f-91b4c8baab66" path="/var/lib/kubelet/pods/b5b961c3-8408-4e91-ac9f-91b4c8baab66/volumes"
Sep 09 20:34:19.643003 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:19.642970    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:19.743721 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:19.743699    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:20.832035 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:20.832004    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:20.932631 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:20.932601    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:21.053912 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:21.053879    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:21.154592 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:21.154522    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:24.036017 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:24.035973    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:24.136941 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:24.136909    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:26.257619 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:26.257586    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:27.097051 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:27.097019    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:27.196599 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:27.196568    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:28.014215 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:28.014178    2550 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" podUID="bcde0f8e-13e1-4cb8-bb02-f963c359b8a2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 09 20:34:28.136768 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:28.136739    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:28.237780 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:28.237752    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:28.294416 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:28.294344    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:36.055159 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:36.055124    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:36.155398 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:36.155363    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:38.014666 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:38.014627    2550 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" podUID="bcde0f8e-13e1-4cb8-bb02-f963c359b8a2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 09 20:34:38.015030 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:38.014705    2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc"
Sep 09 20:34:38.015155 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:38.015139    2550 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"aea17081fc7be981b4a6302f49b68cc51118d1a2434e327b3d37aa72663c47c9"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 09 20:34:38.015219 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:38.015192    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" podUID="bcde0f8e-13e1-4cb8-bb02-f963c359b8a2" containerName="service-proxy" containerID="cri-o://aea17081fc7be981b4a6302f49b68cc51118d1a2434e327b3d37aa72663c47c9" gracePeriod=30
Sep 09 20:34:38.902582 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:38.902550    2550 generic.go:356] "Generic (PLEG): container finished" podID="bcde0f8e-13e1-4cb8-bb02-f963c359b8a2" containerID="aea17081fc7be981b4a6302f49b68cc51118d1a2434e327b3d37aa72663c47c9" exitCode=2
Sep 09 20:34:38.902770 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:38.902625    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" event={"ID":"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2","Type":"ContainerDied","Data":"aea17081fc7be981b4a6302f49b68cc51118d1a2434e327b3d37aa72663c47c9"}
Sep 09 20:34:38.902770 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:34:38.902678    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f896dd9d7-v2nfc" event={"ID":"bcde0f8e-13e1-4cb8-bb02-f963c359b8a2","Type":"ContainerStarted","Data":"6fdc7fa1bb3ad563aaf9ad52ab590b4d3da0554b65163bd5a67d862cb85ceb66"}
Sep 09 20:35:01.052127 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:35:01.052072    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:01.482357 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:35:01.482304    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:10.411289 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:35:10.411236    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:10.473769 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:35:10.473732    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:21.450213 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:35:21.450154    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:21.506879 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:35:21.506847    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:21.564626 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:35:21.564599    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:41.964310 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:35:41.964268    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:42.020372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:35:42.020321    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:01.348147 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:01.348094    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:16.289079 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:16.289034    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:16.341959 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:16.341925    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:28.238297 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:28.238259    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:28.297718 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:28.297687    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:36.069463 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:36.069431    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:36.163498 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:36.163458    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:54.379161 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:54.379122    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:54.441374 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:54.441339    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:54.522377 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:54.522345    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:54.573431 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:54.573406    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:55.138001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:55.137970    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:55.193853 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:36:55.193827    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:37:04.849929 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:37:04.849896    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:37:04.951109 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:37:04.951084    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:37:05.006639 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:37:05.006613    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:37:20.223394 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:37:20.223364    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:37:20.226558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:37:20.226535    2550 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 20:37:20.229835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:37:20.229817    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:38:57.495875 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:38:57.495844    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:58.989009 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:38:58.988975    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:19.251394 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:19.251362    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:20.206649 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:20.206614    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:22.868732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:22.868693    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:22.965959 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:22.965924    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:24.783135 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:24.783100    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:24.921061 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:24.921029    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:25.589227 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.589192    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"]
Sep 09 20:39:25.589438 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.589427    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5b961c3-8408-4e91-ac9f-91b4c8baab66" containerName="registry"
Sep 09 20:39:25.589479 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.589440    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5b961c3-8408-4e91-ac9f-91b4c8baab66" containerName="registry"
Sep 09 20:39:25.589512 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.589479    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5b961c3-8408-4e91-ac9f-91b4c8baab66" containerName="registry"
Sep 09 20:39:25.592308 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.592292    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:25.594334 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.594314    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 20:39:25.594730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.594710    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ml86x\""
Sep 09 20:39:25.594842 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.594709    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 20:39:25.601356 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.601335    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"]
Sep 09 20:39:25.621157 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.621135    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:25.621262 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.621205    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drqq2\" (UniqueName: \"kubernetes.io/projected/0dc15852-9a68-441a-bf91-71c638026a22-kube-api-access-drqq2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:25.621262 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.621230    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:25.721450 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.721421    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drqq2\" (UniqueName: \"kubernetes.io/projected/0dc15852-9a68-441a-bf91-71c638026a22-kube-api-access-drqq2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:25.721450 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.721453    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:25.721618 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.721480    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:25.721872 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.721853    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:25.721907 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.721868    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:25.729750 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.729729    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drqq2\" (UniqueName: \"kubernetes.io/projected/0dc15852-9a68-441a-bf91-71c638026a22-kube-api-access-drqq2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:25.902126 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:25.902042    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:26.168986 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:26.168919    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"]
Sep 09 20:39:26.171741 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:39:26.171714    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dc15852_9a68_441a_bf91_71c638026a22.slice/crio-5916c93d1608a88b715ef36a06edc7e9b51fad56b0be853483ba92516e0da2cc WatchSource:0}: Error finding container 5916c93d1608a88b715ef36a06edc7e9b51fad56b0be853483ba92516e0da2cc: Status 404 returned error can't find the container with id 5916c93d1608a88b715ef36a06edc7e9b51fad56b0be853483ba92516e0da2cc
Sep 09 20:39:26.173689 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:26.173675    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:39:26.804302 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:26.804264    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q" event={"ID":"0dc15852-9a68-441a-bf91-71c638026a22","Type":"ContainerStarted","Data":"5916c93d1608a88b715ef36a06edc7e9b51fad56b0be853483ba92516e0da2cc"}
Sep 09 20:39:27.170040 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:27.169957    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:27.374140 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:27.374104    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:27.486756 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:27.486730    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:29.842843 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:29.842794    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:29.908118 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:29.908091    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:30.777528 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:30.777492    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:30.836495 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:30.836460    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:33.115990 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.115960    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"]
Sep 09 20:39:33.118901 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.118886    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"
Sep 09 20:39:33.124964 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.121179    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 09 20:39:33.125453 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.125430    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 09 20:39:33.125571 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.125521    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dp6nf\""
Sep 09 20:39:33.128822 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.128801    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"]
Sep 09 20:39:33.175238 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.175151    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3edcdd9f-1e12-4ac2-9275-12516263ff24-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-w2pxs\" (UID: \"3edcdd9f-1e12-4ac2-9275-12516263ff24\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"
Sep 09 20:39:33.175238 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.175199    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgbfv\" (UniqueName: \"kubernetes.io/projected/3edcdd9f-1e12-4ac2-9275-12516263ff24-kube-api-access-pgbfv\") pod \"cert-manager-cainjector-559cdd8c87-w2pxs\" (UID: \"3edcdd9f-1e12-4ac2-9275-12516263ff24\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"
Sep 09 20:39:33.276202 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.276152    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3edcdd9f-1e12-4ac2-9275-12516263ff24-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-w2pxs\" (UID: \"3edcdd9f-1e12-4ac2-9275-12516263ff24\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"
Sep 09 20:39:33.276334 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.276207    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgbfv\" (UniqueName: \"kubernetes.io/projected/3edcdd9f-1e12-4ac2-9275-12516263ff24-kube-api-access-pgbfv\") pod \"cert-manager-cainjector-559cdd8c87-w2pxs\" (UID: \"3edcdd9f-1e12-4ac2-9275-12516263ff24\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"
Sep 09 20:39:33.283803 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.283767    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgbfv\" (UniqueName: \"kubernetes.io/projected/3edcdd9f-1e12-4ac2-9275-12516263ff24-kube-api-access-pgbfv\") pod \"cert-manager-cainjector-559cdd8c87-w2pxs\" (UID: \"3edcdd9f-1e12-4ac2-9275-12516263ff24\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"
Sep 09 20:39:33.284953 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.284932    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3edcdd9f-1e12-4ac2-9275-12516263ff24-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-w2pxs\" (UID: \"3edcdd9f-1e12-4ac2-9275-12516263ff24\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"
Sep 09 20:39:33.432107 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.432035    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"
Sep 09 20:39:33.698970 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.698901    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs"]
Sep 09 20:39:33.701356 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:39:33.701331    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3edcdd9f_1e12_4ac2_9275_12516263ff24.slice/crio-853598b016f3f37aea3d404e6928d7726aedb01c488b1fd4ab0a388c7af9b2c5 WatchSource:0}: Error finding container 853598b016f3f37aea3d404e6928d7726aedb01c488b1fd4ab0a388c7af9b2c5: Status 404 returned error can't find the container with id 853598b016f3f37aea3d404e6928d7726aedb01c488b1fd4ab0a388c7af9b2c5
Sep 09 20:39:33.831548 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.831514    2550 generic.go:356] "Generic (PLEG): container finished" podID="0dc15852-9a68-441a-bf91-71c638026a22" containerID="c342d87f114fbd3f88f7cc130c895faa6c7a7858a80f1641aa2f60a28b6ef8e2" exitCode=0
Sep 09 20:39:33.831722 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.831610    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q" event={"ID":"0dc15852-9a68-441a-bf91-71c638026a22","Type":"ContainerDied","Data":"c342d87f114fbd3f88f7cc130c895faa6c7a7858a80f1641aa2f60a28b6ef8e2"}
Sep 09 20:39:33.832936 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:33.832909    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs" event={"ID":"3edcdd9f-1e12-4ac2-9275-12516263ff24","Type":"ContainerStarted","Data":"853598b016f3f37aea3d404e6928d7726aedb01c488b1fd4ab0a388c7af9b2c5"}
Sep 09 20:39:34.767035 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:34.766839    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:37.851034 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:37.850950    2550 generic.go:356] "Generic (PLEG): container finished" podID="0dc15852-9a68-441a-bf91-71c638026a22" containerID="fd92381830b1cc29f7419505994375a524a3e30dd3512a7ddeee6f9dab637d90" exitCode=0
Sep 09 20:39:37.851034 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:37.851031    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q" event={"ID":"0dc15852-9a68-441a-bf91-71c638026a22","Type":"ContainerDied","Data":"fd92381830b1cc29f7419505994375a524a3e30dd3512a7ddeee6f9dab637d90"}
Sep 09 20:39:37.852720 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:37.852694    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs" event={"ID":"3edcdd9f-1e12-4ac2-9275-12516263ff24","Type":"ContainerStarted","Data":"c3b6cfee60666521dd1229a0dab4e412fa0a63889b74ee72b786b955ac9708b9"}
Sep 09 20:39:37.882224 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:37.882154    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w2pxs" podStartSLOduration=1.045148243 podStartE2EDuration="4.882138204s" totalImagesPullingTime="3.836989961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:39:37.881375273 +0000 UTC m=+438.032047617" watchObservedRunningTime="2026-09-09 20:39:37.882138204 +0000 UTC m=+438.032810551"
Sep 09 20:39:38.316683 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:38.316643    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:38.384184 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:38.384138    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:41.506215 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:41.506181    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:41.606681 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:41.606641    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:41.663010 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:41.662970    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:44.880062 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:44.880006    2550 generic.go:356] "Generic (PLEG): container finished" podID="0dc15852-9a68-441a-bf91-71c638026a22" containerID="1f2a22f33fcbc4bf97167c2472cf3cea9293c30fb12a48b9cadd9426d47adb57" exitCode=0
Sep 09 20:39:44.880062 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:44.880073    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q" event={"ID":"0dc15852-9a68-441a-bf91-71c638026a22","Type":"ContainerDied","Data":"1f2a22f33fcbc4bf97167c2472cf3cea9293c30fb12a48b9cadd9426d47adb57"}
Sep 09 20:39:45.997679 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:45.997659    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:46.078713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.078673    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-bundle\") pod \"0dc15852-9a68-441a-bf91-71c638026a22\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") "
Sep 09 20:39:46.078870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.078732    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-util\") pod \"0dc15852-9a68-441a-bf91-71c638026a22\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") "
Sep 09 20:39:46.078870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.078807    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drqq2\" (UniqueName: \"kubernetes.io/projected/0dc15852-9a68-441a-bf91-71c638026a22-kube-api-access-drqq2\") pod \"0dc15852-9a68-441a-bf91-71c638026a22\" (UID: \"0dc15852-9a68-441a-bf91-71c638026a22\") "
Sep 09 20:39:46.079132 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.079105    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-bundle" (OuterVolumeSpecName: "bundle") pod "0dc15852-9a68-441a-bf91-71c638026a22" (UID: "0dc15852-9a68-441a-bf91-71c638026a22"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:39:46.080861 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.080835    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dc15852-9a68-441a-bf91-71c638026a22-kube-api-access-drqq2" (OuterVolumeSpecName: "kube-api-access-drqq2") pod "0dc15852-9a68-441a-bf91-71c638026a22" (UID: "0dc15852-9a68-441a-bf91-71c638026a22"). InnerVolumeSpecName "kube-api-access-drqq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:39:46.084628 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.084604    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-util" (OuterVolumeSpecName: "util") pod "0dc15852-9a68-441a-bf91-71c638026a22" (UID: "0dc15852-9a68-441a-bf91-71c638026a22"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:39:46.179818 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.179766    2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-bundle\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:39:46.179818 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.179786    2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dc15852-9a68-441a-bf91-71c638026a22-util\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:39:46.179818 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.179794    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drqq2\" (UniqueName: \"kubernetes.io/projected/0dc15852-9a68-441a-bf91-71c638026a22-kube-api-access-drqq2\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:39:46.888104 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.888066    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q" event={"ID":"0dc15852-9a68-441a-bf91-71c638026a22","Type":"ContainerDied","Data":"5916c93d1608a88b715ef36a06edc7e9b51fad56b0be853483ba92516e0da2cc"}
Sep 09 20:39:46.888104 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.888086    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frqf9q"
Sep 09 20:39:46.888315 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:46.888119    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5916c93d1608a88b715ef36a06edc7e9b51fad56b0be853483ba92516e0da2cc"
Sep 09 20:39:47.585253 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:47.585217    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:47.685720 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:47.685689    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:54.079967 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:54.079928    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:54.182037 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:54.181999    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:55.520181 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.520135    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"]
Sep 09 20:39:55.520670 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.520463    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dc15852-9a68-441a-bf91-71c638026a22" containerName="extract"
Sep 09 20:39:55.520670 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.520480    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dc15852-9a68-441a-bf91-71c638026a22" containerName="extract"
Sep 09 20:39:55.520670 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.520503    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dc15852-9a68-441a-bf91-71c638026a22" containerName="util"
Sep 09 20:39:55.520670 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.520511    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dc15852-9a68-441a-bf91-71c638026a22" containerName="util"
Sep 09 20:39:55.520670 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.520520    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dc15852-9a68-441a-bf91-71c638026a22" containerName="pull"
Sep 09 20:39:55.520670 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.520527    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dc15852-9a68-441a-bf91-71c638026a22" containerName="pull"
Sep 09 20:39:55.520670 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.520598    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dc15852-9a68-441a-bf91-71c638026a22" containerName="extract"
Sep 09 20:39:55.525027 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.525007    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:55.526956 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.526930    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ml86x\""
Sep 09 20:39:55.526956 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.526948    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 20:39:55.527413 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.527394    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 20:39:55.530657 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.530639    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"]
Sep 09 20:39:55.639651 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.639625    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:55.639802 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.639662    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:55.639802 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.639690    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n44sf\" (UniqueName: \"kubernetes.io/projected/a1add915-2343-412c-9fac-1adc0646a55f-kube-api-access-n44sf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:55.740806 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.740774    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:55.740973 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.740820    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:55.740973 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.740851    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n44sf\" (UniqueName: \"kubernetes.io/projected/a1add915-2343-412c-9fac-1adc0646a55f-kube-api-access-n44sf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:55.741253 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.741231    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:55.741253 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.741246    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:55.748813 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.748784    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n44sf\" (UniqueName: \"kubernetes.io/projected/a1add915-2343-412c-9fac-1adc0646a55f-kube-api-access-n44sf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:55.834504 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:55.834432    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:39:56.100772 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:56.100692    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"]
Sep 09 20:39:56.103478 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:39:56.103450    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1add915_2343_412c_9fac_1adc0646a55f.slice/crio-45e03140c718af2ce1b416f74720bf9f4083adfa479ef0a80b5be1a784b500bf WatchSource:0}: Error finding container 45e03140c718af2ce1b416f74720bf9f4083adfa479ef0a80b5be1a784b500bf: Status 404 returned error can't find the container with id 45e03140c718af2ce1b416f74720bf9f4083adfa479ef0a80b5be1a784b500bf
Sep 09 20:39:56.367583 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:56.367496    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:56.925665 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:56.925632    2550 generic.go:356] "Generic (PLEG): container finished" podID="a1add915-2343-412c-9fac-1adc0646a55f" containerID="f95ffac85cec4c8bfa71e2332dbf2578ad9331b46c457d9cd348370f2f2103fd" exitCode=0
Sep 09 20:39:56.926067 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:56.925711    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p" event={"ID":"a1add915-2343-412c-9fac-1adc0646a55f","Type":"ContainerDied","Data":"f95ffac85cec4c8bfa71e2332dbf2578ad9331b46c457d9cd348370f2f2103fd"}
Sep 09 20:39:56.926067 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:56.925776    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p" event={"ID":"a1add915-2343-412c-9fac-1adc0646a55f","Type":"ContainerStarted","Data":"45e03140c718af2ce1b416f74720bf9f4083adfa479ef0a80b5be1a784b500bf"}
Sep 09 20:39:57.930721 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:57.930641    2550 generic.go:356] "Generic (PLEG): container finished" podID="a1add915-2343-412c-9fac-1adc0646a55f" containerID="8587ea5a2c56d8b54aeda455d8bd0b612ebf6e10b0eec20c636237dbbf268865" exitCode=0
Sep 09 20:39:57.930721 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:57.930701    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p" event={"ID":"a1add915-2343-412c-9fac-1adc0646a55f","Type":"ContainerDied","Data":"8587ea5a2c56d8b54aeda455d8bd0b612ebf6e10b0eec20c636237dbbf268865"}
Sep 09 20:39:58.936011 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:58.935977    2550 generic.go:356] "Generic (PLEG): container finished" podID="a1add915-2343-412c-9fac-1adc0646a55f" containerID="42e6705a9587dcd20e82dd9fd400d183b7eaa670b691ad08a12df681271f89d9" exitCode=0
Sep 09 20:39:58.936385 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:39:58.936040    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p" event={"ID":"a1add915-2343-412c-9fac-1adc0646a55f","Type":"ContainerDied","Data":"42e6705a9587dcd20e82dd9fd400d183b7eaa670b691ad08a12df681271f89d9"}
Sep 09 20:40:00.051505 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.051483    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:40:00.173077 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.173049    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-util\") pod \"a1add915-2343-412c-9fac-1adc0646a55f\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") "
Sep 09 20:40:00.173077 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.173082    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-bundle\") pod \"a1add915-2343-412c-9fac-1adc0646a55f\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") "
Sep 09 20:40:00.173329 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.173105    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n44sf\" (UniqueName: \"kubernetes.io/projected/a1add915-2343-412c-9fac-1adc0646a55f-kube-api-access-n44sf\") pod \"a1add915-2343-412c-9fac-1adc0646a55f\" (UID: \"a1add915-2343-412c-9fac-1adc0646a55f\") "
Sep 09 20:40:00.173749 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.173722    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-bundle" (OuterVolumeSpecName: "bundle") pod "a1add915-2343-412c-9fac-1adc0646a55f" (UID: "a1add915-2343-412c-9fac-1adc0646a55f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:40:00.175095 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.175059    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1add915-2343-412c-9fac-1adc0646a55f-kube-api-access-n44sf" (OuterVolumeSpecName: "kube-api-access-n44sf") pod "a1add915-2343-412c-9fac-1adc0646a55f" (UID: "a1add915-2343-412c-9fac-1adc0646a55f"). InnerVolumeSpecName "kube-api-access-n44sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:40:00.178614 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.178591    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-util" (OuterVolumeSpecName: "util") pod "a1add915-2343-412c-9fac-1adc0646a55f" (UID: "a1add915-2343-412c-9fac-1adc0646a55f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:40:00.274038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.274017    2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-util\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:00.274038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.274036    2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1add915-2343-412c-9fac-1adc0646a55f-bundle\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:00.274152 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.274046    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n44sf\" (UniqueName: \"kubernetes.io/projected/a1add915-2343-412c-9fac-1adc0646a55f-kube-api-access-n44sf\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:00.588388 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.588318    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:00.676725 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.676691    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:00.945518 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.945436    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p"
Sep 09 20:40:00.945660 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.945438    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk4r2p" event={"ID":"a1add915-2343-412c-9fac-1adc0646a55f","Type":"ContainerDied","Data":"45e03140c718af2ce1b416f74720bf9f4083adfa479ef0a80b5be1a784b500bf"}
Sep 09 20:40:00.945660 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:00.945552    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45e03140c718af2ce1b416f74720bf9f4083adfa479ef0a80b5be1a784b500bf"
Sep 09 20:40:11.754491 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:11.754452    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:12.341391 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.341360    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:12.559235 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.559203    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:12.664044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.663978    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"]
Sep 09 20:40:12.664209 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.664197    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1add915-2343-412c-9fac-1adc0646a55f" containerName="pull"
Sep 09 20:40:12.664255 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.664211    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1add915-2343-412c-9fac-1adc0646a55f" containerName="pull"
Sep 09 20:40:12.664255 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.664227    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1add915-2343-412c-9fac-1adc0646a55f" containerName="util"
Sep 09 20:40:12.664255 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.664232    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1add915-2343-412c-9fac-1adc0646a55f" containerName="util"
Sep 09 20:40:12.664255 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.664239    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1add915-2343-412c-9fac-1adc0646a55f" containerName="extract"
Sep 09 20:40:12.664255 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.664244    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1add915-2343-412c-9fac-1adc0646a55f" containerName="extract"
Sep 09 20:40:12.664399 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.664281    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1add915-2343-412c-9fac-1adc0646a55f" containerName="extract"
Sep 09 20:40:12.666175 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.666140    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.668026 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.668003    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 20:40:12.668129 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.668025    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p8mdg\""
Sep 09 20:40:12.668940 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.668925    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 20:40:12.669053 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.669036    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 20:40:12.669153 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.669138    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 20:40:12.686676 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.686652    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"]
Sep 09 20:40:12.760294 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.760270    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/394ec89c-e336-4d95-a554-5e7080fe9862-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.760652 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.760323    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2znfc\" (UniqueName: \"kubernetes.io/projected/394ec89c-e336-4d95-a554-5e7080fe9862-kube-api-access-2znfc\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.760652 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.760410    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/394ec89c-e336-4d95-a554-5e7080fe9862-tmp\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.760652 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.760456    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/394ec89c-e336-4d95-a554-5e7080fe9862-webhook-cert\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.760652 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.760488    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/394ec89c-e336-4d95-a554-5e7080fe9862-manifests\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.794700 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.794678    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:12.861062 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.861035    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/394ec89c-e336-4d95-a554-5e7080fe9862-manifests\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.861150 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.861066    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/394ec89c-e336-4d95-a554-5e7080fe9862-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.861150 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.861098    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2znfc\" (UniqueName: \"kubernetes.io/projected/394ec89c-e336-4d95-a554-5e7080fe9862-kube-api-access-2znfc\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.861150 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.861126    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/394ec89c-e336-4d95-a554-5e7080fe9862-tmp\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.861305 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.861155    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/394ec89c-e336-4d95-a554-5e7080fe9862-webhook-cert\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.861512 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.861488    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/394ec89c-e336-4d95-a554-5e7080fe9862-manifests\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.863469 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.863442    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/394ec89c-e336-4d95-a554-5e7080fe9862-tmp\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.863550 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.863523    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/394ec89c-e336-4d95-a554-5e7080fe9862-webhook-cert\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.863589 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.863560    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/394ec89c-e336-4d95-a554-5e7080fe9862-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.877413 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.877394    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2znfc\" (UniqueName: \"kubernetes.io/projected/394ec89c-e336-4d95-a554-5e7080fe9862-kube-api-access-2znfc\") pod \"opendatahub-operator-controller-manager-859cf57674-5fdk2\" (UID: \"394ec89c-e336-4d95-a554-5e7080fe9862\") " pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.895005 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.894979    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:12.976013 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.975984    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:12.995658 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:12.995632    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:13.095961 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:13.095927    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:13.196705 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:13.196666    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:13.251924 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:13.251905    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"]
Sep 09 20:40:13.253863 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:40:13.253839    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod394ec89c_e336_4d95_a554_5e7080fe9862.slice/crio-27d38c5890000e24edd20b88546abf076102863f22b51a0ae69b03bdb65cfc0c WatchSource:0}: Error finding container 27d38c5890000e24edd20b88546abf076102863f22b51a0ae69b03bdb65cfc0c: Status 404 returned error can't find the container with id 27d38c5890000e24edd20b88546abf076102863f22b51a0ae69b03bdb65cfc0c
Sep 09 20:40:13.990019 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:13.989984    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2" event={"ID":"394ec89c-e336-4d95-a554-5e7080fe9862","Type":"ContainerStarted","Data":"27d38c5890000e24edd20b88546abf076102863f22b51a0ae69b03bdb65cfc0c"}
Sep 09 20:40:17.003755 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:17.003716    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2" event={"ID":"394ec89c-e336-4d95-a554-5e7080fe9862","Type":"ContainerStarted","Data":"83eea7af5bb6ab180ee2f6e33b798940e2a79e1e03254a1edb81bcb116a819bb"}
Sep 09 20:40:18.011923 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:18.011836    2550 generic.go:356] "Generic (PLEG): container finished" podID="394ec89c-e336-4d95-a554-5e7080fe9862" containerID="83eea7af5bb6ab180ee2f6e33b798940e2a79e1e03254a1edb81bcb116a819bb" exitCode=0
Sep 09 20:40:18.012275 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:18.011912    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2" event={"ID":"394ec89c-e336-4d95-a554-5e7080fe9862","Type":"ContainerDied","Data":"83eea7af5bb6ab180ee2f6e33b798940e2a79e1e03254a1edb81bcb116a819bb"}
Sep 09 20:40:19.017092 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:19.017050    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2" event={"ID":"394ec89c-e336-4d95-a554-5e7080fe9862","Type":"ContainerStarted","Data":"5e3a1ebe23e2f490123a554482544cb703baab70773d3d1377669fa86c3af66a"}
Sep 09 20:40:19.017467 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:19.017181    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:19.038451 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:19.038405    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2" podStartSLOduration=2.310734208 podStartE2EDuration="7.038392136s" totalImagesPullingTime="3.641634414s" totalInitContainerRuntime="1.086023514s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:40:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:19.037249492 +0000 UTC m=+479.187921836" watchObservedRunningTime="2026-09-09 20:40:19.038392136 +0000 UTC m=+479.189064479"
Sep 09 20:40:25.819545 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:25.819508    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:26.929572 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:26.929540    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"]
Sep 09 20:40:26.937924 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:26.937904    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:26.940199 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:26.940160    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 20:40:26.940740 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:26.940718    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 20:40:26.940864 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:26.940785    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ml86x\""
Sep 09 20:40:26.943002 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:26.942981    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"]
Sep 09 20:40:27.058809 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.058784    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:27.058939 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.058813    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:27.058939 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.058846    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkvkr\" (UniqueName: \"kubernetes.io/projected/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-kube-api-access-mkvkr\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:27.159099 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.159065    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:27.159215 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.159119    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkvkr\" (UniqueName: \"kubernetes.io/projected/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-kube-api-access-mkvkr\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:27.159319 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.159297    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:27.159482 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.159467    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:27.159581 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.159564    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:27.166889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.166867    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkvkr\" (UniqueName: \"kubernetes.io/projected/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-kube-api-access-mkvkr\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:27.248329 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.248308    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:27.514690 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:27.514609    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"]
Sep 09 20:40:27.517319 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:40:27.517292    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca76b3e_1b05_4f19_8f56_d4c3ee17197b.slice/crio-7a7e9787e7d0f449d0d9cb0ad9f2866009f9bb29a5d904724d03f571f15b46c2 WatchSource:0}: Error finding container 7a7e9787e7d0f449d0d9cb0ad9f2866009f9bb29a5d904724d03f571f15b46c2: Status 404 returned error can't find the container with id 7a7e9787e7d0f449d0d9cb0ad9f2866009f9bb29a5d904724d03f571f15b46c2
Sep 09 20:40:28.049680 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:28.049645    2550 generic.go:356] "Generic (PLEG): container finished" podID="dca76b3e-1b05-4f19-8f56-d4c3ee17197b" containerID="b2500467c7b6670ffd9842d4d56bfdd87c13c6b727ac2a74a6d7508967ab8392" exitCode=0
Sep 09 20:40:28.050274 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:28.049703    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l" event={"ID":"dca76b3e-1b05-4f19-8f56-d4c3ee17197b","Type":"ContainerDied","Data":"b2500467c7b6670ffd9842d4d56bfdd87c13c6b727ac2a74a6d7508967ab8392"}
Sep 09 20:40:28.050274 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:28.049752    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l" event={"ID":"dca76b3e-1b05-4f19-8f56-d4c3ee17197b","Type":"ContainerStarted","Data":"7a7e9787e7d0f449d0d9cb0ad9f2866009f9bb29a5d904724d03f571f15b46c2"}
Sep 09 20:40:29.055006 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:29.054935    2550 generic.go:356] "Generic (PLEG): container finished" podID="dca76b3e-1b05-4f19-8f56-d4c3ee17197b" containerID="43ff84c5da4a20538d5740cebe38d9c785daf1c7b0d85f358bc89b0e234af486" exitCode=0
Sep 09 20:40:29.055006 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:29.054973    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l" event={"ID":"dca76b3e-1b05-4f19-8f56-d4c3ee17197b","Type":"ContainerDied","Data":"43ff84c5da4a20538d5740cebe38d9c785daf1c7b0d85f358bc89b0e234af486"}
Sep 09 20:40:30.022558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:30.022528    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-859cf57674-5fdk2"
Sep 09 20:40:30.061869 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:30.061839    2550 generic.go:356] "Generic (PLEG): container finished" podID="dca76b3e-1b05-4f19-8f56-d4c3ee17197b" containerID="9c7e0dcc1bd6220c2a0624293df755ea8577b9df586fe60cb5f3354d8c157061" exitCode=0
Sep 09 20:40:30.062249 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:30.061915    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l" event={"ID":"dca76b3e-1b05-4f19-8f56-d4c3ee17197b","Type":"ContainerDied","Data":"9c7e0dcc1bd6220c2a0624293df755ea8577b9df586fe60cb5f3354d8c157061"}
Sep 09 20:40:31.182004 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.181981    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:31.288922 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.288895    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-util\") pod \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") "
Sep 09 20:40:31.289069 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.288950    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkvkr\" (UniqueName: \"kubernetes.io/projected/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-kube-api-access-mkvkr\") pod \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") "
Sep 09 20:40:31.289069 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.288980    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-bundle\") pod \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\" (UID: \"dca76b3e-1b05-4f19-8f56-d4c3ee17197b\") "
Sep 09 20:40:31.289682 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.289645    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-bundle" (OuterVolumeSpecName: "bundle") pod "dca76b3e-1b05-4f19-8f56-d4c3ee17197b" (UID: "dca76b3e-1b05-4f19-8f56-d4c3ee17197b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:40:31.290966 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.290936    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-kube-api-access-mkvkr" (OuterVolumeSpecName: "kube-api-access-mkvkr") pod "dca76b3e-1b05-4f19-8f56-d4c3ee17197b" (UID: "dca76b3e-1b05-4f19-8f56-d4c3ee17197b"). InnerVolumeSpecName "kube-api-access-mkvkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:40:31.294246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.294224    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-util" (OuterVolumeSpecName: "util") pod "dca76b3e-1b05-4f19-8f56-d4c3ee17197b" (UID: "dca76b3e-1b05-4f19-8f56-d4c3ee17197b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:40:31.389388 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.389336    2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-util\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:31.389388 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.389356    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkvkr\" (UniqueName: \"kubernetes.io/projected/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-kube-api-access-mkvkr\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:31.389388 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.389366    2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dca76b3e-1b05-4f19-8f56-d4c3ee17197b-bundle\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:31.598452 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.598421    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:31.788145 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:31.788103    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:32.015860 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.015834    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:32.034367 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.034343    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"]
Sep 09 20:40:32.034576 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.034563    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca76b3e-1b05-4f19-8f56-d4c3ee17197b" containerName="extract"
Sep 09 20:40:32.034612 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.034578    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca76b3e-1b05-4f19-8f56-d4c3ee17197b" containerName="extract"
Sep 09 20:40:32.034612 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.034601    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca76b3e-1b05-4f19-8f56-d4c3ee17197b" containerName="pull"
Sep 09 20:40:32.034612 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.034606    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca76b3e-1b05-4f19-8f56-d4c3ee17197b" containerName="pull"
Sep 09 20:40:32.034701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.034617    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca76b3e-1b05-4f19-8f56-d4c3ee17197b" containerName="util"
Sep 09 20:40:32.034701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.034623    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca76b3e-1b05-4f19-8f56-d4c3ee17197b" containerName="util"
Sep 09 20:40:32.034701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.034660    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="dca76b3e-1b05-4f19-8f56-d4c3ee17197b" containerName="extract"
Sep 09 20:40:32.038520 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.038450    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.044914 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.044890    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 09 20:40:32.045028 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.044967    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 20:40:32.045028 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.044978    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 09 20:40:32.050518 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.050502    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 20:40:32.071335 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.071314    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l"
Sep 09 20:40:32.071426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.071331    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ldc4l" event={"ID":"dca76b3e-1b05-4f19-8f56-d4c3ee17197b","Type":"ContainerDied","Data":"7a7e9787e7d0f449d0d9cb0ad9f2866009f9bb29a5d904724d03f571f15b46c2"}
Sep 09 20:40:32.071426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.071363    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a7e9787e7d0f449d0d9cb0ad9f2866009f9bb29a5d904724d03f571f15b46c2"
Sep 09 20:40:32.093313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.093254    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"]
Sep 09 20:40:32.194409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.194386    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgthc\" (UniqueName: \"kubernetes.io/projected/296ba124-bae7-497c-bf5b-1331e85f4b77-kube-api-access-cgthc\") pod \"kube-auth-proxy-65d657c8d5-cww2l\" (UID: \"296ba124-bae7-497c-bf5b-1331e85f4b77\") " pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.194710 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.194419    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/296ba124-bae7-497c-bf5b-1331e85f4b77-tmp\") pod \"kube-auth-proxy-65d657c8d5-cww2l\" (UID: \"296ba124-bae7-497c-bf5b-1331e85f4b77\") " pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.194710 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.194439    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296ba124-bae7-497c-bf5b-1331e85f4b77-tls-certs\") pod \"kube-auth-proxy-65d657c8d5-cww2l\" (UID: \"296ba124-bae7-497c-bf5b-1331e85f4b77\") " pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.215877 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.215844    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:32.294958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.294888    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgthc\" (UniqueName: \"kubernetes.io/projected/296ba124-bae7-497c-bf5b-1331e85f4b77-kube-api-access-cgthc\") pod \"kube-auth-proxy-65d657c8d5-cww2l\" (UID: \"296ba124-bae7-497c-bf5b-1331e85f4b77\") " pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.294958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.294927    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/296ba124-bae7-497c-bf5b-1331e85f4b77-tmp\") pod \"kube-auth-proxy-65d657c8d5-cww2l\" (UID: \"296ba124-bae7-497c-bf5b-1331e85f4b77\") " pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.294958 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.294956    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296ba124-bae7-497c-bf5b-1331e85f4b77-tls-certs\") pod \"kube-auth-proxy-65d657c8d5-cww2l\" (UID: \"296ba124-bae7-497c-bf5b-1331e85f4b77\") " pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.297200 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.297154    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/296ba124-bae7-497c-bf5b-1331e85f4b77-tmp\") pod \"kube-auth-proxy-65d657c8d5-cww2l\" (UID: \"296ba124-bae7-497c-bf5b-1331e85f4b77\") " pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.297321 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.297304    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296ba124-bae7-497c-bf5b-1331e85f4b77-tls-certs\") pod \"kube-auth-proxy-65d657c8d5-cww2l\" (UID: \"296ba124-bae7-497c-bf5b-1331e85f4b77\") " pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.310335 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.310312    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgthc\" (UniqueName: \"kubernetes.io/projected/296ba124-bae7-497c-bf5b-1331e85f4b77-kube-api-access-cgthc\") pod \"kube-auth-proxy-65d657c8d5-cww2l\" (UID: \"296ba124-bae7-497c-bf5b-1331e85f4b77\") " pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.347206 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.347186    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"
Sep 09 20:40:32.666077 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:32.666006    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l"]
Sep 09 20:40:32.669229 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:40:32.669206    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod296ba124_bae7_497c_bf5b_1331e85f4b77.slice/crio-ae40b7ae22c98aae64937bac7bc1c23b53dad8cc3a97d3f97fc9ca485613c10d WatchSource:0}: Error finding container ae40b7ae22c98aae64937bac7bc1c23b53dad8cc3a97d3f97fc9ca485613c10d: Status 404 returned error can't find the container with id ae40b7ae22c98aae64937bac7bc1c23b53dad8cc3a97d3f97fc9ca485613c10d
Sep 09 20:40:33.076551 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.076513    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l" event={"ID":"296ba124-bae7-497c-bf5b-1331e85f4b77","Type":"ContainerStarted","Data":"ae40b7ae22c98aae64937bac7bc1c23b53dad8cc3a97d3f97fc9ca485613c10d"}
Sep 09 20:40:33.761570 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.761520    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:33.765943 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.765912    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"]
Sep 09 20:40:33.769460 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.769436    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:33.771741 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.771425    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-b2crw\""
Sep 09 20:40:33.771741 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.771466    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 20:40:33.771741 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.771673    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 20:40:33.771741 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.771781    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 20:40:33.772184 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.771912    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 20:40:33.784232 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.784155    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"]
Sep 09 20:40:33.860839 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.860804    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:33.907846 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.907802    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px4sp\" (UniqueName: \"kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-kube-api-access-px4sp\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:33.908024 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.907852    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:33.908024 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.907884    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:33.908024 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.907952    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:33.908024 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.908007    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:33.908284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.908092    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:33.908284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:33.908135    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/38bf4582-0755-499a-a83d-1ebbf3fe60e4-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.008901 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.008851    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.009085 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.008910    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/38bf4582-0755-499a-a83d-1ebbf3fe60e4-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.009085 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.008936    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px4sp\" (UniqueName: \"kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-kube-api-access-px4sp\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.009413 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.009314    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.009413 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.009352    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.009413 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.009389    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.009619 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.009442    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.009619 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.009589    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.012614 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.012543    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.012723 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.012648    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.012797 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.012755    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.013261 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.013239    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/38bf4582-0755-499a-a83d-1ebbf3fe60e4-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.017180 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.017138    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.017537 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.017516    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px4sp\" (UniqueName: \"kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-kube-api-access-px4sp\") pod \"istiod-openshift-gateway-7b7995d5cd-hmc5c\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.081119 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.081086    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:34.827368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:34.827310    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:35.540095 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:35.540026    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"]
Sep 09 20:40:35.545699 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:40:35.545666    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bf4582_0755_499a_a83d_1ebbf3fe60e4.slice/crio-0d260321c162dfad0d9085dcd97d0e9c521e2c891215fbe5ca0f4a85d730ec90 WatchSource:0}: Error finding container 0d260321c162dfad0d9085dcd97d0e9c521e2c891215fbe5ca0f4a85d730ec90: Status 404 returned error can't find the container with id 0d260321c162dfad0d9085dcd97d0e9c521e2c891215fbe5ca0f4a85d730ec90
Sep 09 20:40:36.090055 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:36.090017    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l" event={"ID":"296ba124-bae7-497c-bf5b-1331e85f4b77","Type":"ContainerStarted","Data":"6d9032a415afd53cf6e67a7f2f3b86a59cdf8b806237755e58961290599bcdef"}
Sep 09 20:40:36.091196 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:36.091153    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c" event={"ID":"38bf4582-0755-499a-a83d-1ebbf3fe60e4","Type":"ContainerStarted","Data":"0d260321c162dfad0d9085dcd97d0e9c521e2c891215fbe5ca0f4a85d730ec90"}
Sep 09 20:40:36.106072 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:36.106022    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65d657c8d5-cww2l" podStartSLOduration=1.447823766 podStartE2EDuration="4.106005928s" totalImagesPullingTime="2.658182162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:36.105049658 +0000 UTC m=+496.255722001" watchObservedRunningTime="2026-09-09 20:40:36.106005928 +0000 UTC m=+496.256678270"
Sep 09 20:40:38.512614 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:38.512571    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:40:38.512873 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:38.512645    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:40:39.105859 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.105814    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c" event={"ID":"38bf4582-0755-499a-a83d-1ebbf3fe60e4","Type":"ContainerStarted","Data":"99f6d8b1d0e971717b5e946d3830b65eea0d65b20b7da4434e897c0b93f31fbf"}
Sep 09 20:40:39.106041 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.105998    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:39.107764 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.107739    2550 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-hmc5c container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 20:40:39.107870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.107794    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c" podUID="38bf4582-0755-499a-a83d-1ebbf3fe60e4" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 20:40:39.226001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.225958    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:39.323225 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.323191    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:39.871116 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.871052    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c" podStartSLOduration=3.90658142 podStartE2EDuration="6.871028133s" totalImagesPullingTime="2.964446713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:39.125959048 +0000 UTC m=+499.276631403" watchObservedRunningTime="2026-09-09 20:40:39.871028133 +0000 UTC m=+500.021700476"
Sep 09 20:40:39.872335 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.872312    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"]
Sep 09 20:40:39.875537 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.875519    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:39.877499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.877479    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-kf5fn\""
Sep 09 20:40:39.887464 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.887446    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"]
Sep 09 20:40:39.901063 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.901037    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:39.954355 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.954325    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:39.954520 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.954362    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vt4s\" (UniqueName: \"kubernetes.io/projected/b5d19ecc-2831-40b5-a6fa-9f444be087f8-kube-api-access-5vt4s\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:39.954520 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.954408    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:39.954520 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.954435    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:39.954520 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.954453    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:39.954520 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.954468    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:39.954520 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.954496    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:39.954745 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.954540    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:39.954745 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:39.954568    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055023 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.054991    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055216 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055037    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vt4s\" (UniqueName: \"kubernetes.io/projected/b5d19ecc-2831-40b5-a6fa-9f444be087f8-kube-api-access-5vt4s\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055216 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055091    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055216 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055115    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055216 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055184    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055436 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055228    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055436 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055266    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055436 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055303    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055436 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055329    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055938 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055534    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055938 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055654    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055938 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055783    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.055938 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.055831    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.056331 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.056306    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.057814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.057791    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.057814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.057801    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.063445 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.063418    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5d19ecc-2831-40b5-a6fa-9f444be087f8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.063629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.063609    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vt4s\" (UniqueName: \"kubernetes.io/projected/b5d19ecc-2831-40b5-a6fa-9f444be087f8-kube-api-access-5vt4s\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c24thp\" (UID: \"b5d19ecc-2831-40b5-a6fa-9f444be087f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.104377 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.104351    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:40.109950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.109931    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:40.185272 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.185200    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:40.244004 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.243950    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:40.460295 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:40.460272    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"]
Sep 09 20:40:40.462751 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:40:40.462724    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d19ecc_2831_40b5_a6fa_9f444be087f8.slice/crio-4c0f51046512adb9a9ee4dabe0e0fbcdf9143ab1ac12ad90f65c6f6cb0620c3c WatchSource:0}: Error finding container 4c0f51046512adb9a9ee4dabe0e0fbcdf9143ab1ac12ad90f65c6f6cb0620c3c: Status 404 returned error can't find the container with id 4c0f51046512adb9a9ee4dabe0e0fbcdf9143ab1ac12ad90f65c6f6cb0620c3c
Sep 09 20:40:41.114029 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:41.113992    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp" event={"ID":"b5d19ecc-2831-40b5-a6fa-9f444be087f8","Type":"ContainerStarted","Data":"4c0f51046512adb9a9ee4dabe0e0fbcdf9143ab1ac12ad90f65c6f6cb0620c3c"}
Sep 09 20:40:42.856801 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:42.856761    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:42.920678 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:42.920647    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:43.020181 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:43.020124    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:43.116138 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:43.116105    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:40:43.116250 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:43.116181    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:40:43.116250 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:43.116212    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:40:43.318359 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:43.318326    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:43.788475 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:43.788439    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:43.859317 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:43.859286    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:44.127890 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:44.127808    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp" event={"ID":"b5d19ecc-2831-40b5-a6fa-9f444be087f8","Type":"ContainerStarted","Data":"f1ca42d0fe5c9e72013d964df77b63effe07774903783bfe1c1b51f030bfeaf6"}
Sep 09 20:40:44.151581 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:44.151533    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp" podStartSLOduration=2.500822189 podStartE2EDuration="5.151519189s" totalImagesPullingTime="2.650697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:44.14566392 +0000 UTC m=+504.296336263" watchObservedRunningTime="2026-09-09 20:40:44.151519189 +0000 UTC m=+504.302191530"
Sep 09 20:40:44.185639 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:44.185616    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:44.190491 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:44.190471    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:45.131859 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:45.131821    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:45.132684 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:45.132664    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c24thp"
Sep 09 20:40:53.224013 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.223970    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:53.267173 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.267132    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"]
Sep 09 20:40:53.277090 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.277064    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"
Sep 09 20:40:53.280381 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.280363    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-rxr7h\""
Sep 09 20:40:53.280507 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.280492    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 09 20:40:53.283398 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.283375    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"]
Sep 09 20:40:53.286082 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.286064    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"]
Sep 09 20:40:53.289347 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.289331    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"
Sep 09 20:40:53.292597 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.292576    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 09 20:40:53.293243 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.293227    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lvf5w\""
Sep 09 20:40:53.308881 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.308859    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"]
Sep 09 20:40:53.346188 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.346143    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebf3005c-ac3e-4904-a0ea-33326e1979f0-cert\") pod \"kserve-controller-manager-d6f4b65b8-9tm2k\" (UID: \"ebf3005c-ac3e-4904-a0ea-33326e1979f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"
Sep 09 20:40:53.346329 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.346215    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4bpz\" (UniqueName: \"kubernetes.io/projected/61c527f5-57d1-404a-bffa-bcbb1433d443-kube-api-access-p4bpz\") pod \"llmisvc-controller-manager-777c6bd7d5-dscmd\" (UID: \"61c527f5-57d1-404a-bffa-bcbb1433d443\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"
Sep 09 20:40:53.346329 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.346235    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c527f5-57d1-404a-bffa-bcbb1433d443-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dscmd\" (UID: \"61c527f5-57d1-404a-bffa-bcbb1433d443\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"
Sep 09 20:40:53.346329 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.346263    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr9gg\" (UniqueName: \"kubernetes.io/projected/ebf3005c-ac3e-4904-a0ea-33326e1979f0-kube-api-access-kr9gg\") pod \"kserve-controller-manager-d6f4b65b8-9tm2k\" (UID: \"ebf3005c-ac3e-4904-a0ea-33326e1979f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"
Sep 09 20:40:53.349644 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.349613    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:53.446799 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.446766    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr9gg\" (UniqueName: \"kubernetes.io/projected/ebf3005c-ac3e-4904-a0ea-33326e1979f0-kube-api-access-kr9gg\") pod \"kserve-controller-manager-d6f4b65b8-9tm2k\" (UID: \"ebf3005c-ac3e-4904-a0ea-33326e1979f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"
Sep 09 20:40:53.446996 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.446815    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebf3005c-ac3e-4904-a0ea-33326e1979f0-cert\") pod \"kserve-controller-manager-d6f4b65b8-9tm2k\" (UID: \"ebf3005c-ac3e-4904-a0ea-33326e1979f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"
Sep 09 20:40:53.446996 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.446855    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4bpz\" (UniqueName: \"kubernetes.io/projected/61c527f5-57d1-404a-bffa-bcbb1433d443-kube-api-access-p4bpz\") pod \"llmisvc-controller-manager-777c6bd7d5-dscmd\" (UID: \"61c527f5-57d1-404a-bffa-bcbb1433d443\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"
Sep 09 20:40:53.446996 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.446872    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c527f5-57d1-404a-bffa-bcbb1433d443-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dscmd\" (UID: \"61c527f5-57d1-404a-bffa-bcbb1433d443\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"
Sep 09 20:40:53.449356 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.449335    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c527f5-57d1-404a-bffa-bcbb1433d443-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dscmd\" (UID: \"61c527f5-57d1-404a-bffa-bcbb1433d443\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"
Sep 09 20:40:53.449448 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.449371    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebf3005c-ac3e-4904-a0ea-33326e1979f0-cert\") pod \"kserve-controller-manager-d6f4b65b8-9tm2k\" (UID: \"ebf3005c-ac3e-4904-a0ea-33326e1979f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"
Sep 09 20:40:53.464047 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.464016    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr9gg\" (UniqueName: \"kubernetes.io/projected/ebf3005c-ac3e-4904-a0ea-33326e1979f0-kube-api-access-kr9gg\") pod \"kserve-controller-manager-d6f4b65b8-9tm2k\" (UID: \"ebf3005c-ac3e-4904-a0ea-33326e1979f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"
Sep 09 20:40:53.464149 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.464057    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4bpz\" (UniqueName: \"kubernetes.io/projected/61c527f5-57d1-404a-bffa-bcbb1433d443-kube-api-access-p4bpz\") pod \"llmisvc-controller-manager-777c6bd7d5-dscmd\" (UID: \"61c527f5-57d1-404a-bffa-bcbb1433d443\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"
Sep 09 20:40:53.557942 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.557867    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:53.588759 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.588730    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"
Sep 09 20:40:53.599870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.599846    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"
Sep 09 20:40:53.858452 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.858362    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"]
Sep 09 20:40:53.861825 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:40:53.861798    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf3005c_ac3e_4904_a0ea_33326e1979f0.slice/crio-a64fde00d517bbf52295d95f62ac89c833237e2c728bf26a5de2a037bf287771 WatchSource:0}: Error finding container a64fde00d517bbf52295d95f62ac89c833237e2c728bf26a5de2a037bf287771: Status 404 returned error can't find the container with id a64fde00d517bbf52295d95f62ac89c833237e2c728bf26a5de2a037bf287771
Sep 09 20:40:53.877873 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:53.877851    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"]
Sep 09 20:40:53.879997 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:40:53.879973    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod61c527f5_57d1_404a_bffa_bcbb1433d443.slice/crio-3d13085f615b031d58e96085142be0410ce859552e1af5aa58afd774c66c8ecc WatchSource:0}: Error finding container 3d13085f615b031d58e96085142be0410ce859552e1af5aa58afd774c66c8ecc: Status 404 returned error can't find the container with id 3d13085f615b031d58e96085142be0410ce859552e1af5aa58afd774c66c8ecc
Sep 09 20:40:54.171464 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:54.171378    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k" event={"ID":"ebf3005c-ac3e-4904-a0ea-33326e1979f0","Type":"ContainerStarted","Data":"a64fde00d517bbf52295d95f62ac89c833237e2c728bf26a5de2a037bf287771"}
Sep 09 20:40:54.172565 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:54.172542    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd" event={"ID":"61c527f5-57d1-404a-bffa-bcbb1433d443","Type":"ContainerStarted","Data":"3d13085f615b031d58e96085142be0410ce859552e1af5aa58afd774c66c8ecc"}
Sep 09 20:40:54.511909 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:54.511876    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:54.623957 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:54.623920    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:54.757823 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:54.757789    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:54.833733 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:54.833655    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"]
Sep 09 20:40:54.833919 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:54.833894    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c" podUID="38bf4582-0755-499a-a83d-1ebbf3fe60e4" containerName="discovery" containerID="cri-o://99f6d8b1d0e971717b5e946d3830b65eea0d65b20b7da4434e897c0b93f31fbf" gracePeriod=30
Sep 09 20:40:55.180953 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.180922    2550 generic.go:356] "Generic (PLEG): container finished" podID="38bf4582-0755-499a-a83d-1ebbf3fe60e4" containerID="99f6d8b1d0e971717b5e946d3830b65eea0d65b20b7da4434e897c0b93f31fbf" exitCode=0
Sep 09 20:40:55.181084 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.180999    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c" event={"ID":"38bf4582-0755-499a-a83d-1ebbf3fe60e4","Type":"ContainerDied","Data":"99f6d8b1d0e971717b5e946d3830b65eea0d65b20b7da4434e897c0b93f31fbf"}
Sep 09 20:40:55.181084 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.181062    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c" event={"ID":"38bf4582-0755-499a-a83d-1ebbf3fe60e4","Type":"ContainerDied","Data":"0d260321c162dfad0d9085dcd97d0e9c521e2c891215fbe5ca0f4a85d730ec90"}
Sep 09 20:40:55.181084 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.181081    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d260321c162dfad0d9085dcd97d0e9c521e2c891215fbe5ca0f4a85d730ec90"
Sep 09 20:40:55.189642 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.189618    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:55.259941 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.259913    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-token\") pod \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") "
Sep 09 20:40:55.259941 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.259945    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-kubeconfig\") pod \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") "
Sep 09 20:40:55.260430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.259967    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-dns-cert\") pod \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") "
Sep 09 20:40:55.260430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.260005    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/38bf4582-0755-499a-a83d-1ebbf3fe60e4-local-certs\") pod \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") "
Sep 09 20:40:55.260430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.260047    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-ca-configmap\") pod \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") "
Sep 09 20:40:55.260430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.260080    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px4sp\" (UniqueName: \"kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-kube-api-access-px4sp\") pod \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") "
Sep 09 20:40:55.260430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.260157    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-cacerts\") pod \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\" (UID: \"38bf4582-0755-499a-a83d-1ebbf3fe60e4\") "
Sep 09 20:40:55.260748 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.260701    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "38bf4582-0755-499a-a83d-1ebbf3fe60e4" (UID: "38bf4582-0755-499a-a83d-1ebbf3fe60e4"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:40:55.262805 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.262759    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "38bf4582-0755-499a-a83d-1ebbf3fe60e4" (UID: "38bf4582-0755-499a-a83d-1ebbf3fe60e4"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:40:55.263027 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.262987    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-kube-api-access-px4sp" (OuterVolumeSpecName: "kube-api-access-px4sp") pod "38bf4582-0755-499a-a83d-1ebbf3fe60e4" (UID: "38bf4582-0755-499a-a83d-1ebbf3fe60e4"). InnerVolumeSpecName "kube-api-access-px4sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:40:55.263124 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.263102    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "38bf4582-0755-499a-a83d-1ebbf3fe60e4" (UID: "38bf4582-0755-499a-a83d-1ebbf3fe60e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:40:55.263381 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.263357    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "38bf4582-0755-499a-a83d-1ebbf3fe60e4" (UID: "38bf4582-0755-499a-a83d-1ebbf3fe60e4"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:40:55.264024 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.263964    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-cacerts" (OuterVolumeSpecName: "cacerts") pod "38bf4582-0755-499a-a83d-1ebbf3fe60e4" (UID: "38bf4582-0755-499a-a83d-1ebbf3fe60e4"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:40:55.264108 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.264080    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bf4582-0755-499a-a83d-1ebbf3fe60e4-local-certs" (OuterVolumeSpecName: "local-certs") pod "38bf4582-0755-499a-a83d-1ebbf3fe60e4" (UID: "38bf4582-0755-499a-a83d-1ebbf3fe60e4"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:40:55.360802 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.360765    2550 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-cacerts\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:55.360802 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.360794    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:55.360802 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.360808    2550 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-kubeconfig\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:55.361103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.360821    2550 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-dns-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:55.361103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.360835    2550 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/38bf4582-0755-499a-a83d-1ebbf3fe60e4-local-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:55.361103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.360847    2550 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/38bf4582-0755-499a-a83d-1ebbf3fe60e4-istio-csr-ca-configmap\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:55.361103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:55.360860    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px4sp\" (UniqueName: \"kubernetes.io/projected/38bf4582-0755-499a-a83d-1ebbf3fe60e4-kube-api-access-px4sp\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:40:56.186187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:56.186135    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"
Sep 09 20:40:56.208061 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:56.208032    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"]
Sep 09 20:40:56.215476 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:56.215451    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c"]
Sep 09 20:40:56.279974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:56.279943    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38bf4582-0755-499a-a83d-1ebbf3fe60e4" path="/var/lib/kubelet/pods/38bf4582-0755-499a-a83d-1ebbf3fe60e4/volumes"
Sep 09 20:40:57.191235 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:57.191206    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k" event={"ID":"ebf3005c-ac3e-4904-a0ea-33326e1979f0","Type":"ContainerStarted","Data":"a0a2920e4339fde4b1f6a2b18066cb99c8a535b173422dbf11cb0968591024f9"}
Sep 09 20:40:57.191622 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:57.191271    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"
Sep 09 20:40:57.192669 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:57.192644    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd" event={"ID":"61c527f5-57d1-404a-bffa-bcbb1433d443","Type":"ContainerStarted","Data":"f32559b0034e62e1ac36f4deb06f29b12ef505d448a74c91719274d25b6f9af6"}
Sep 09 20:40:57.192756 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:57.192723    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"
Sep 09 20:40:57.213128 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:57.213088    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k" podStartSLOduration=1.056451591 podStartE2EDuration="4.213074386s" totalImagesPullingTime="3.156622795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:57.211869879 +0000 UTC m=+517.362542225" watchObservedRunningTime="2026-09-09 20:40:57.213074386 +0000 UTC m=+517.363746728"
Sep 09 20:40:57.235967 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:40:57.235917    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd" podStartSLOduration=1.102389142 podStartE2EDuration="4.235899917s" totalImagesPullingTime="3.133510775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:57.234521798 +0000 UTC m=+517.385194153" watchObservedRunningTime="2026-09-09 20:40:57.235899917 +0000 UTC m=+517.386572261"
Sep 09 20:41:00.110268 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:00.110229    2550 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-hmc5c container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.19:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 20:41:00.110632 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:00.110294    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hmc5c" podUID="38bf4582-0755-499a-a83d-1ebbf3fe60e4" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.19:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 20:41:03.490690 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:03.490658    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:28.198895 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:28.198864    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dscmd"
Sep 09 20:41:28.201677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:28.201652    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9tm2k"
Sep 09 20:41:43.784471 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:43.784435    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:43.885930 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:43.885893    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:47.709931 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.709895    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"]
Sep 09 20:41:47.710420 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.710301    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38bf4582-0755-499a-a83d-1ebbf3fe60e4" containerName="discovery"
Sep 09 20:41:47.710420 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.710317    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="38bf4582-0755-499a-a83d-1ebbf3fe60e4" containerName="discovery"
Sep 09 20:41:47.710420 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.710396    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="38bf4582-0755-499a-a83d-1ebbf3fe60e4" containerName="discovery"
Sep 09 20:41:47.714745 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.714718    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:47.717750 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.717719    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 20:41:47.718003 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.717988    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 20:41:47.718077 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.717992    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ml86x\""
Sep 09 20:41:47.722387 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.722079    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"]
Sep 09 20:41:47.801820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.801780    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"]
Sep 09 20:41:47.804942 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.804921    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:47.812713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.812691    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"]
Sep 09 20:41:47.851488 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.851452    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp5df\" (UniqueName: \"kubernetes.io/projected/c7f3684d-388b-4546-bd50-0294658107cd-kube-api-access-mp5df\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:47.851627 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.851506    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:47.851627 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.851543    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:47.906222 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.906193    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"]
Sep 09 20:41:47.909243 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.909229    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:47.918801 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.918780    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"]
Sep 09 20:41:47.952093 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.952059    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:47.952263 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.952100    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp5df\" (UniqueName: \"kubernetes.io/projected/c7f3684d-388b-4546-bd50-0294658107cd-kube-api-access-mp5df\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:47.952263 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.952192    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:47.952263 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.952243    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:47.952434 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.952299    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:47.952434 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.952331    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5jmk\" (UniqueName: \"kubernetes.io/projected/86abb40d-fb99-469d-b546-508b1f958975-kube-api-access-q5jmk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:47.952605 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.952587    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:47.952639 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.952626    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:47.960809 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:47.960766    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp5df\" (UniqueName: \"kubernetes.io/projected/c7f3684d-388b-4546-bd50-0294658107cd-kube-api-access-mp5df\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:48.015007 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.014979    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"]
Sep 09 20:41:48.018243 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.018225    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.024373 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.024237    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:48.047029 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.047005    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"]
Sep 09 20:41:48.053501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.053477    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5jmk\" (UniqueName: \"kubernetes.io/projected/86abb40d-fb99-469d-b546-508b1f958975-kube-api-access-q5jmk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:48.053637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.053546    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:48.053637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.053579    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:48.053637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.053605    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:48.053637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.053629    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:48.053830 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.053667    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvcm5\" (UniqueName: \"kubernetes.io/projected/360828d1-d6cf-4449-ae76-dd2095f39690-kube-api-access-dvcm5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:48.054089 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.054069    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:48.054197 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.054112    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:48.086492 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.086429    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5jmk\" (UniqueName: \"kubernetes.io/projected/86abb40d-fb99-469d-b546-508b1f958975-kube-api-access-q5jmk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:48.114155 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.114133    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:48.154809 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.154777    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.154959 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.154893    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl6wc\" (UniqueName: \"kubernetes.io/projected/9744a1f1-b28c-469a-a762-afab8cea0dfc-kube-api-access-cl6wc\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.155019 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.154965    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:48.155019 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.154994    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:48.155124 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.155035    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvcm5\" (UniqueName: \"kubernetes.io/projected/360828d1-d6cf-4449-ae76-dd2095f39690-kube-api-access-dvcm5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:48.155124 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.155063    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.155306 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.155288    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:48.155399 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.155377    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:48.164729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.164703    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvcm5\" (UniqueName: \"kubernetes.io/projected/360828d1-d6cf-4449-ae76-dd2095f39690-kube-api-access-dvcm5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:48.217902 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.217836    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:48.256095 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.256063    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.256276 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.256110    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.256276 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.256182    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl6wc\" (UniqueName: \"kubernetes.io/projected/9744a1f1-b28c-469a-a762-afab8cea0dfc-kube-api-access-cl6wc\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.256535 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.256508    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.256601 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.256519    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.265732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.265675    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl6wc\" (UniqueName: \"kubernetes.io/projected/9744a1f1-b28c-469a-a762-afab8cea0dfc-kube-api-access-cl6wc\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.293959 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.293932    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"]
Sep 09 20:41:48.295830 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:41:48.295805    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f3684d_388b_4546_bd50_0294658107cd.slice/crio-eb16d484baabbff9d79bd00cf68fdc982a936453284ce0974916028961b6ec3b WatchSource:0}: Error finding container eb16d484baabbff9d79bd00cf68fdc982a936453284ce0974916028961b6ec3b: Status 404 returned error can't find the container with id eb16d484baabbff9d79bd00cf68fdc982a936453284ce0974916028961b6ec3b
Sep 09 20:41:48.326853 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.326828    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:48.387484 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.387359    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"]
Sep 09 20:41:48.391431 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:41:48.389552    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86abb40d_fb99_469d_b546_508b1f958975.slice/crio-2eb5e371c051f32d9a1810ca66767eb9bc85c0c9d13db093510a427a6bf63baa WatchSource:0}: Error finding container 2eb5e371c051f32d9a1810ca66767eb9bc85c0c9d13db093510a427a6bf63baa: Status 404 returned error can't find the container with id 2eb5e371c051f32d9a1810ca66767eb9bc85c0c9d13db093510a427a6bf63baa
Sep 09 20:41:48.400486 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.400458    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt" event={"ID":"c7f3684d-388b-4546-bd50-0294658107cd","Type":"ContainerStarted","Data":"699d60a9a79b93d4ad6afea7e8898c4efa647f44c7ed02516732f44de197f626"}
Sep 09 20:41:48.400610 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.400510    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt" event={"ID":"c7f3684d-388b-4546-bd50-0294658107cd","Type":"ContainerStarted","Data":"eb16d484baabbff9d79bd00cf68fdc982a936453284ce0974916028961b6ec3b"}
Sep 09 20:41:48.401841 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.401808    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k" event={"ID":"86abb40d-fb99-469d-b546-508b1f958975","Type":"ContainerStarted","Data":"2eb5e371c051f32d9a1810ca66767eb9bc85c0c9d13db093510a427a6bf63baa"}
Sep 09 20:41:48.487295 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.487270    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"]
Sep 09 20:41:48.488464 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:41:48.488442    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod360828d1_d6cf_4449_ae76_dd2095f39690.slice/crio-e43455206f8379d579eeeb3b0e0e128411ad954f738a24a96d712e5f68c6fc4a WatchSource:0}: Error finding container e43455206f8379d579eeeb3b0e0e128411ad954f738a24a96d712e5f68c6fc4a: Status 404 returned error can't find the container with id e43455206f8379d579eeeb3b0e0e128411ad954f738a24a96d712e5f68c6fc4a
Sep 09 20:41:48.604530 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:48.604504    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"]
Sep 09 20:41:48.643914 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:41:48.643891    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9744a1f1_b28c_469a_a762_afab8cea0dfc.slice/crio-2c18e4a6034840f76c8915607fd4bb512ed03be5166ecdff1033a6c45893cfdf WatchSource:0}: Error finding container 2c18e4a6034840f76c8915607fd4bb512ed03be5166ecdff1033a6c45893cfdf: Status 404 returned error can't find the container with id 2c18e4a6034840f76c8915607fd4bb512ed03be5166ecdff1033a6c45893cfdf
Sep 09 20:41:49.407219 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:49.407187    2550 generic.go:356] "Generic (PLEG): container finished" podID="c7f3684d-388b-4546-bd50-0294658107cd" containerID="699d60a9a79b93d4ad6afea7e8898c4efa647f44c7ed02516732f44de197f626" exitCode=0
Sep 09 20:41:49.407741 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:49.407256    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt" event={"ID":"c7f3684d-388b-4546-bd50-0294658107cd","Type":"ContainerDied","Data":"699d60a9a79b93d4ad6afea7e8898c4efa647f44c7ed02516732f44de197f626"}
Sep 09 20:41:49.408792 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:49.408768    2550 generic.go:356] "Generic (PLEG): container finished" podID="9744a1f1-b28c-469a-a762-afab8cea0dfc" containerID="ba9a7e6fc4d2601155664370b27c1657744745740752fc85acecf03956322a13" exitCode=0
Sep 09 20:41:49.408885 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:49.408855    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr" event={"ID":"9744a1f1-b28c-469a-a762-afab8cea0dfc","Type":"ContainerDied","Data":"ba9a7e6fc4d2601155664370b27c1657744745740752fc85acecf03956322a13"}
Sep 09 20:41:49.408944 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:49.408898    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr" event={"ID":"9744a1f1-b28c-469a-a762-afab8cea0dfc","Type":"ContainerStarted","Data":"2c18e4a6034840f76c8915607fd4bb512ed03be5166ecdff1033a6c45893cfdf"}
Sep 09 20:41:49.410417 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:49.410397    2550 generic.go:356] "Generic (PLEG): container finished" podID="86abb40d-fb99-469d-b546-508b1f958975" containerID="a682652676fc56a71b9e314553bcafdc09dfb6509f5b1e1e1a657be2fccc09bb" exitCode=0
Sep 09 20:41:49.410482 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:49.410469    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k" event={"ID":"86abb40d-fb99-469d-b546-508b1f958975","Type":"ContainerDied","Data":"a682652676fc56a71b9e314553bcafdc09dfb6509f5b1e1e1a657be2fccc09bb"}
Sep 09 20:41:49.412317 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:49.412213    2550 generic.go:356] "Generic (PLEG): container finished" podID="360828d1-d6cf-4449-ae76-dd2095f39690" containerID="6f645f4bae32cc9c9a5b7b9b18c0c0a0a7085c5e3538f27af56246de76acaf61" exitCode=0
Sep 09 20:41:49.412317 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:49.412261    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2" event={"ID":"360828d1-d6cf-4449-ae76-dd2095f39690","Type":"ContainerDied","Data":"6f645f4bae32cc9c9a5b7b9b18c0c0a0a7085c5e3538f27af56246de76acaf61"}
Sep 09 20:41:49.412317 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:49.412311    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2" event={"ID":"360828d1-d6cf-4449-ae76-dd2095f39690","Type":"ContainerStarted","Data":"e43455206f8379d579eeeb3b0e0e128411ad954f738a24a96d712e5f68c6fc4a"}
Sep 09 20:41:50.418753 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:50.418719    2550 generic.go:356] "Generic (PLEG): container finished" podID="c7f3684d-388b-4546-bd50-0294658107cd" containerID="359292e141855b099c471888f16c604dcf616516179e4298105fc9d68b31218b" exitCode=0
Sep 09 20:41:50.419101 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:50.418797    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt" event={"ID":"c7f3684d-388b-4546-bd50-0294658107cd","Type":"ContainerDied","Data":"359292e141855b099c471888f16c604dcf616516179e4298105fc9d68b31218b"}
Sep 09 20:41:50.420808 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:50.420778    2550 generic.go:356] "Generic (PLEG): container finished" podID="86abb40d-fb99-469d-b546-508b1f958975" containerID="55f4e2130ff3d055c3a3a9d52ac13dcba2c9fb9354790d35546f714fd1608ce7" exitCode=0
Sep 09 20:41:50.420890 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:50.420828    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k" event={"ID":"86abb40d-fb99-469d-b546-508b1f958975","Type":"ContainerDied","Data":"55f4e2130ff3d055c3a3a9d52ac13dcba2c9fb9354790d35546f714fd1608ce7"}
Sep 09 20:41:51.427503 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:51.427424    2550 generic.go:356] "Generic (PLEG): container finished" podID="c7f3684d-388b-4546-bd50-0294658107cd" containerID="cbd98ee020cbf232d2a91e603f4b002dd8ff1b8a8a96e49ccdc5e5b1a94a50c6" exitCode=0
Sep 09 20:41:51.427503 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:51.427491    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt" event={"ID":"c7f3684d-388b-4546-bd50-0294658107cd","Type":"ContainerDied","Data":"cbd98ee020cbf232d2a91e603f4b002dd8ff1b8a8a96e49ccdc5e5b1a94a50c6"}
Sep 09 20:41:51.429190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:51.429155    2550 generic.go:356] "Generic (PLEG): container finished" podID="9744a1f1-b28c-469a-a762-afab8cea0dfc" containerID="8b89a31296c9f0d1e06927bfe1a740da3b1e5a1fb54ac0026b06fdc45a7e8ea8" exitCode=0
Sep 09 20:41:51.429300 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:51.429256    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr" event={"ID":"9744a1f1-b28c-469a-a762-afab8cea0dfc","Type":"ContainerDied","Data":"8b89a31296c9f0d1e06927bfe1a740da3b1e5a1fb54ac0026b06fdc45a7e8ea8"}
Sep 09 20:41:51.431374 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:51.431350    2550 generic.go:356] "Generic (PLEG): container finished" podID="86abb40d-fb99-469d-b546-508b1f958975" containerID="2212c2345cf4ffb47a6196307ed177ed4aed2b2bc33791a5bc19372ca32f91d8" exitCode=0
Sep 09 20:41:51.431483 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:51.431426    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k" event={"ID":"86abb40d-fb99-469d-b546-508b1f958975","Type":"ContainerDied","Data":"2212c2345cf4ffb47a6196307ed177ed4aed2b2bc33791a5bc19372ca32f91d8"}
Sep 09 20:41:51.433103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:51.433083    2550 generic.go:356] "Generic (PLEG): container finished" podID="360828d1-d6cf-4449-ae76-dd2095f39690" containerID="bb329edb6fb4d0a2201112f71b323d9afd5171653f1432bc6a9073795f85063c" exitCode=0
Sep 09 20:41:51.433196 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:51.433123    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2" event={"ID":"360828d1-d6cf-4449-ae76-dd2095f39690","Type":"ContainerDied","Data":"bb329edb6fb4d0a2201112f71b323d9afd5171653f1432bc6a9073795f85063c"}
Sep 09 20:41:52.439910 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.439879    2550 generic.go:356] "Generic (PLEG): container finished" podID="9744a1f1-b28c-469a-a762-afab8cea0dfc" containerID="878a3bb77e23467a38ec075c7c3280d5d709d75460d3f46ae99c33551b45beb9" exitCode=0
Sep 09 20:41:52.440367 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.439965    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr" event={"ID":"9744a1f1-b28c-469a-a762-afab8cea0dfc","Type":"ContainerDied","Data":"878a3bb77e23467a38ec075c7c3280d5d709d75460d3f46ae99c33551b45beb9"}
Sep 09 20:41:52.441915 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.441891    2550 generic.go:356] "Generic (PLEG): container finished" podID="360828d1-d6cf-4449-ae76-dd2095f39690" containerID="0ae39889c6968a35702395bc27998860a5ea25bc66fc53127c4ca82a148e0641" exitCode=0
Sep 09 20:41:52.442043 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.441961    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2" event={"ID":"360828d1-d6cf-4449-ae76-dd2095f39690","Type":"ContainerDied","Data":"0ae39889c6968a35702395bc27998860a5ea25bc66fc53127c4ca82a148e0641"}
Sep 09 20:41:52.565515 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.565491    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:52.592132 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.592112    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:52.694871 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.694794    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp5df\" (UniqueName: \"kubernetes.io/projected/c7f3684d-388b-4546-bd50-0294658107cd-kube-api-access-mp5df\") pod \"c7f3684d-388b-4546-bd50-0294658107cd\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") "
Sep 09 20:41:52.694871 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.694841    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-bundle\") pod \"86abb40d-fb99-469d-b546-508b1f958975\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") "
Sep 09 20:41:52.695055 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.694879    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5jmk\" (UniqueName: \"kubernetes.io/projected/86abb40d-fb99-469d-b546-508b1f958975-kube-api-access-q5jmk\") pod \"86abb40d-fb99-469d-b546-508b1f958975\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") "
Sep 09 20:41:52.695055 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.694905    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-util\") pod \"86abb40d-fb99-469d-b546-508b1f958975\" (UID: \"86abb40d-fb99-469d-b546-508b1f958975\") "
Sep 09 20:41:52.695055 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.694946    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-bundle\") pod \"c7f3684d-388b-4546-bd50-0294658107cd\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") "
Sep 09 20:41:52.695055 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.694968    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-util\") pod \"c7f3684d-388b-4546-bd50-0294658107cd\" (UID: \"c7f3684d-388b-4546-bd50-0294658107cd\") "
Sep 09 20:41:52.695595 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.695550    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-bundle" (OuterVolumeSpecName: "bundle") pod "c7f3684d-388b-4546-bd50-0294658107cd" (UID: "c7f3684d-388b-4546-bd50-0294658107cd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:52.695835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.695801    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-bundle" (OuterVolumeSpecName: "bundle") pod "86abb40d-fb99-469d-b546-508b1f958975" (UID: "86abb40d-fb99-469d-b546-508b1f958975"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:52.697042 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.697021    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86abb40d-fb99-469d-b546-508b1f958975-kube-api-access-q5jmk" (OuterVolumeSpecName: "kube-api-access-q5jmk") pod "86abb40d-fb99-469d-b546-508b1f958975" (UID: "86abb40d-fb99-469d-b546-508b1f958975"). InnerVolumeSpecName "kube-api-access-q5jmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:41:52.697340 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.697320    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f3684d-388b-4546-bd50-0294658107cd-kube-api-access-mp5df" (OuterVolumeSpecName: "kube-api-access-mp5df") pod "c7f3684d-388b-4546-bd50-0294658107cd" (UID: "c7f3684d-388b-4546-bd50-0294658107cd"). InnerVolumeSpecName "kube-api-access-mp5df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:41:52.701644 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.701623    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-util" (OuterVolumeSpecName: "util") pod "c7f3684d-388b-4546-bd50-0294658107cd" (UID: "c7f3684d-388b-4546-bd50-0294658107cd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:52.703418 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.703394    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-util" (OuterVolumeSpecName: "util") pod "86abb40d-fb99-469d-b546-508b1f958975" (UID: "86abb40d-fb99-469d-b546-508b1f958975"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:52.795756 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.795715    2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-bundle\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:52.795756 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.795754    2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f3684d-388b-4546-bd50-0294658107cd-util\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:52.795756 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.795764    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp5df\" (UniqueName: \"kubernetes.io/projected/c7f3684d-388b-4546-bd50-0294658107cd-kube-api-access-mp5df\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:52.795756 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.795774    2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-bundle\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:52.795998 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.795783    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5jmk\" (UniqueName: \"kubernetes.io/projected/86abb40d-fb99-469d-b546-508b1f958975-kube-api-access-q5jmk\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:52.795998 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:52.795790    2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86abb40d-fb99-469d-b546-508b1f958975-util\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:53.447906 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.447832    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt"
Sep 09 20:41:53.448324 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.447831    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chvqbt" event={"ID":"c7f3684d-388b-4546-bd50-0294658107cd","Type":"ContainerDied","Data":"eb16d484baabbff9d79bd00cf68fdc982a936453284ce0974916028961b6ec3b"}
Sep 09 20:41:53.448324 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.447963    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb16d484baabbff9d79bd00cf68fdc982a936453284ce0974916028961b6ec3b"
Sep 09 20:41:53.449856 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.449837    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k"
Sep 09 20:41:53.449975 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.449867    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4qz4k" event={"ID":"86abb40d-fb99-469d-b546-508b1f958975","Type":"ContainerDied","Data":"2eb5e371c051f32d9a1810ca66767eb9bc85c0c9d13db093510a427a6bf63baa"}
Sep 09 20:41:53.449975 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.449897    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2eb5e371c051f32d9a1810ca66767eb9bc85c0c9d13db093510a427a6bf63baa"
Sep 09 20:41:53.583859 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.583833    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:53.602304 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.602278    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl6wc\" (UniqueName: \"kubernetes.io/projected/9744a1f1-b28c-469a-a762-afab8cea0dfc-kube-api-access-cl6wc\") pod \"9744a1f1-b28c-469a-a762-afab8cea0dfc\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") "
Sep 09 20:41:53.602425 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.602325    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-util\") pod \"9744a1f1-b28c-469a-a762-afab8cea0dfc\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") "
Sep 09 20:41:53.604787 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.604755    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9744a1f1-b28c-469a-a762-afab8cea0dfc-kube-api-access-cl6wc" (OuterVolumeSpecName: "kube-api-access-cl6wc") pod "9744a1f1-b28c-469a-a762-afab8cea0dfc" (UID: "9744a1f1-b28c-469a-a762-afab8cea0dfc"). InnerVolumeSpecName "kube-api-access-cl6wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:41:53.609403 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.609380    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-util" (OuterVolumeSpecName: "util") pod "9744a1f1-b28c-469a-a762-afab8cea0dfc" (UID: "9744a1f1-b28c-469a-a762-afab8cea0dfc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:53.610697 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.610682    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:53.702771 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.702687    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-bundle\") pod \"9744a1f1-b28c-469a-a762-afab8cea0dfc\" (UID: \"9744a1f1-b28c-469a-a762-afab8cea0dfc\") "
Sep 09 20:41:53.702771 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.702723    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvcm5\" (UniqueName: \"kubernetes.io/projected/360828d1-d6cf-4449-ae76-dd2095f39690-kube-api-access-dvcm5\") pod \"360828d1-d6cf-4449-ae76-dd2095f39690\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") "
Sep 09 20:41:53.702771 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.702744    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-bundle\") pod \"360828d1-d6cf-4449-ae76-dd2095f39690\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") "
Sep 09 20:41:53.702771 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.702763    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-util\") pod \"360828d1-d6cf-4449-ae76-dd2095f39690\" (UID: \"360828d1-d6cf-4449-ae76-dd2095f39690\") "
Sep 09 20:41:53.703094 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.702912    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl6wc\" (UniqueName: \"kubernetes.io/projected/9744a1f1-b28c-469a-a762-afab8cea0dfc-kube-api-access-cl6wc\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:53.703094 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.702922    2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-util\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:53.703230 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.703139    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-bundle" (OuterVolumeSpecName: "bundle") pod "360828d1-d6cf-4449-ae76-dd2095f39690" (UID: "360828d1-d6cf-4449-ae76-dd2095f39690"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:53.703313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.703289    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-bundle" (OuterVolumeSpecName: "bundle") pod "9744a1f1-b28c-469a-a762-afab8cea0dfc" (UID: "9744a1f1-b28c-469a-a762-afab8cea0dfc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:53.704830 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.704806    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/360828d1-d6cf-4449-ae76-dd2095f39690-kube-api-access-dvcm5" (OuterVolumeSpecName: "kube-api-access-dvcm5") pod "360828d1-d6cf-4449-ae76-dd2095f39690" (UID: "360828d1-d6cf-4449-ae76-dd2095f39690"). InnerVolumeSpecName "kube-api-access-dvcm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:41:53.708142 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.708111    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-util" (OuterVolumeSpecName: "util") pod "360828d1-d6cf-4449-ae76-dd2095f39690" (UID: "360828d1-d6cf-4449-ae76-dd2095f39690"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:53.803360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.803316    2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9744a1f1-b28c-469a-a762-afab8cea0dfc-bundle\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:53.803360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.803357    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvcm5\" (UniqueName: \"kubernetes.io/projected/360828d1-d6cf-4449-ae76-dd2095f39690-kube-api-access-dvcm5\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:53.803360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.803367    2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-bundle\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:53.803360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:53.803375    2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/360828d1-d6cf-4449-ae76-dd2095f39690-util\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:41:54.455784 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:54.455755    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr"
Sep 09 20:41:54.455784 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:54.455780    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nljxr" event={"ID":"9744a1f1-b28c-469a-a762-afab8cea0dfc","Type":"ContainerDied","Data":"2c18e4a6034840f76c8915607fd4bb512ed03be5166ecdff1033a6c45893cfdf"}
Sep 09 20:41:54.456249 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:54.455825    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c18e4a6034840f76c8915607fd4bb512ed03be5166ecdff1033a6c45893cfdf"
Sep 09 20:41:54.457855 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:54.457830    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2" event={"ID":"360828d1-d6cf-4449-ae76-dd2095f39690","Type":"ContainerDied","Data":"e43455206f8379d579eeeb3b0e0e128411ad954f738a24a96d712e5f68c6fc4a"}
Sep 09 20:41:54.457991 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:54.457871    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e43455206f8379d579eeeb3b0e0e128411ad954f738a24a96d712e5f68c6fc4a"
Sep 09 20:41:54.457991 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:54.457905    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fk8n2"
Sep 09 20:41:54.646682 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:54.646660    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:59.014870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:41:59.014836    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:01.059337 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059298    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"]
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059660    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86abb40d-fb99-469d-b546-508b1f958975" containerName="extract"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059676    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="86abb40d-fb99-469d-b546-508b1f958975" containerName="extract"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059690    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9744a1f1-b28c-469a-a762-afab8cea0dfc" containerName="pull"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059698    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9744a1f1-b28c-469a-a762-afab8cea0dfc" containerName="pull"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059708    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f3684d-388b-4546-bd50-0294658107cd" containerName="util"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059715    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f3684d-388b-4546-bd50-0294658107cd" containerName="util"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059726    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9744a1f1-b28c-469a-a762-afab8cea0dfc" containerName="util"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059734    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9744a1f1-b28c-469a-a762-afab8cea0dfc" containerName="util"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059741    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="360828d1-d6cf-4449-ae76-dd2095f39690" containerName="pull"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059748    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="360828d1-d6cf-4449-ae76-dd2095f39690" containerName="pull"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059758    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86abb40d-fb99-469d-b546-508b1f958975" containerName="util"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059767    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="86abb40d-fb99-469d-b546-508b1f958975" containerName="util"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059776    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f3684d-388b-4546-bd50-0294658107cd" containerName="pull"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059785    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f3684d-388b-4546-bd50-0294658107cd" containerName="pull"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059802    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86abb40d-fb99-469d-b546-508b1f958975" containerName="pull"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059810    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="86abb40d-fb99-469d-b546-508b1f958975" containerName="pull"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059819    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="360828d1-d6cf-4449-ae76-dd2095f39690" containerName="extract"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059827    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="360828d1-d6cf-4449-ae76-dd2095f39690" containerName="extract"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059844    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="360828d1-d6cf-4449-ae76-dd2095f39690" containerName="util"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059851    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="360828d1-d6cf-4449-ae76-dd2095f39690" containerName="util"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059859    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9744a1f1-b28c-469a-a762-afab8cea0dfc" containerName="extract"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059867    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9744a1f1-b28c-469a-a762-afab8cea0dfc" containerName="extract"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059878    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f3684d-388b-4546-bd50-0294658107cd" containerName="extract"
Sep 09 20:42:01.059889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059884    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f3684d-388b-4546-bd50-0294658107cd" containerName="extract"
Sep 09 20:42:01.060962 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059954    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9744a1f1-b28c-469a-a762-afab8cea0dfc" containerName="extract"
Sep 09 20:42:01.060962 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059966    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="360828d1-d6cf-4449-ae76-dd2095f39690" containerName="extract"
Sep 09 20:42:01.060962 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059976    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7f3684d-388b-4546-bd50-0294658107cd" containerName="extract"
Sep 09 20:42:01.060962 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.059988    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="86abb40d-fb99-469d-b546-508b1f958975" containerName="extract"
Sep 09 20:42:01.064053 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.064034    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:01.066342 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.066322    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 20:42:01.066425 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.066373    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rl2zr\""
Sep 09 20:42:01.066425 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.066324    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 20:42:01.081020 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.080998    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"]
Sep 09 20:42:01.152890 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.152867    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dff9j\" (UniqueName: \"kubernetes.io/projected/53248ed7-d834-4bff-8746-aee79273e1f6-kube-api-access-dff9j\") pod \"limitador-operator-controller-manager-74495c69f8-k7lzx\" (UID: \"53248ed7-d834-4bff-8746-aee79273e1f6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:01.253418 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.253389    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dff9j\" (UniqueName: \"kubernetes.io/projected/53248ed7-d834-4bff-8746-aee79273e1f6-kube-api-access-dff9j\") pod \"limitador-operator-controller-manager-74495c69f8-k7lzx\" (UID: \"53248ed7-d834-4bff-8746-aee79273e1f6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:01.261493 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.261466    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dff9j\" (UniqueName: \"kubernetes.io/projected/53248ed7-d834-4bff-8746-aee79273e1f6-kube-api-access-dff9j\") pod \"limitador-operator-controller-manager-74495c69f8-k7lzx\" (UID: \"53248ed7-d834-4bff-8746-aee79273e1f6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:01.373191 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.373120    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:01.647830 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.647806    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"]
Sep 09 20:42:01.650391 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:42:01.650358    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53248ed7_d834_4bff_8746_aee79273e1f6.slice/crio-efb83001bda8cbb4df2027c8ea260cd2e006c52a546cdde47b017fc86721bdad WatchSource:0}: Error finding container efb83001bda8cbb4df2027c8ea260cd2e006c52a546cdde47b017fc86721bdad: Status 404 returned error can't find the container with id efb83001bda8cbb4df2027c8ea260cd2e006c52a546cdde47b017fc86721bdad
Sep 09 20:42:01.838824 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:01.838791    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:02.492453 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:02.492424    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx" event={"ID":"53248ed7-d834-4bff-8746-aee79273e1f6","Type":"ContainerStarted","Data":"efb83001bda8cbb4df2027c8ea260cd2e006c52a546cdde47b017fc86721bdad"}
Sep 09 20:42:03.101465 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:03.101424    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x"]
Sep 09 20:42:03.105078 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:03.105055    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x"
Sep 09 20:42:03.107672 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:03.107651    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 09 20:42:03.108352 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:03.108331    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-httwk\""
Sep 09 20:42:03.117378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:03.117351    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x"]
Sep 09 20:42:03.170371 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:03.170342    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66nhp\" (UniqueName: \"kubernetes.io/projected/62fc8f6b-a719-4dde-968f-e8ea792dfe3b-kube-api-access-66nhp\") pod \"dns-operator-controller-manager-7b84cd7c9c-kjh5x\" (UID: \"62fc8f6b-a719-4dde-968f-e8ea792dfe3b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x"
Sep 09 20:42:03.271214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:03.271160    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66nhp\" (UniqueName: \"kubernetes.io/projected/62fc8f6b-a719-4dde-968f-e8ea792dfe3b-kube-api-access-66nhp\") pod \"dns-operator-controller-manager-7b84cd7c9c-kjh5x\" (UID: \"62fc8f6b-a719-4dde-968f-e8ea792dfe3b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x"
Sep 09 20:42:03.283692 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:03.283650    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66nhp\" (UniqueName: \"kubernetes.io/projected/62fc8f6b-a719-4dde-968f-e8ea792dfe3b-kube-api-access-66nhp\") pod \"dns-operator-controller-manager-7b84cd7c9c-kjh5x\" (UID: \"62fc8f6b-a719-4dde-968f-e8ea792dfe3b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x"
Sep 09 20:42:03.419898 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:03.419833    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x"
Sep 09 20:42:03.617329 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:03.617292    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:04.018587 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:04.018547    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:04.269344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:04.269256    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:04.427841 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:04.427803    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:04.685534 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:04.685506    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x"]
Sep 09 20:42:04.686072 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:42:04.686049    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62fc8f6b_a719_4dde_968f_e8ea792dfe3b.slice/crio-292047c63c0f828859abd31a1bd5aae440c189088bd1b53d6160208039b4fef7 WatchSource:0}: Error finding container 292047c63c0f828859abd31a1bd5aae440c189088bd1b53d6160208039b4fef7: Status 404 returned error can't find the container with id 292047c63c0f828859abd31a1bd5aae440c189088bd1b53d6160208039b4fef7
Sep 09 20:42:05.509736 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:05.509701    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx" event={"ID":"53248ed7-d834-4bff-8746-aee79273e1f6","Type":"ContainerStarted","Data":"9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989"}
Sep 09 20:42:05.509931 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:05.509772    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:05.510991 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:05.510968    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x" event={"ID":"62fc8f6b-a719-4dde-968f-e8ea792dfe3b","Type":"ContainerStarted","Data":"292047c63c0f828859abd31a1bd5aae440c189088bd1b53d6160208039b4fef7"}
Sep 09 20:42:05.552094 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:05.552042    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx" podStartSLOduration=1.721010272 podStartE2EDuration="4.552026044s" totalImagesPullingTime="2.831015772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:05.550740245 +0000 UTC m=+585.701412588" watchObservedRunningTime="2026-09-09 20:42:05.552026044 +0000 UTC m=+585.702698387"
Sep 09 20:42:05.573713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:05.573678    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:06.576989 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:06.576953    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:06.690764 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:06.690725    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:07.523712 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:07.523679    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x" event={"ID":"62fc8f6b-a719-4dde-968f-e8ea792dfe3b","Type":"ContainerStarted","Data":"940583ec06e6ab9f9a6f49bb2f8eb70ae5064a5f00eba49399fcc625109b1e23"}
Sep 09 20:42:07.523881 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:07.523763    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x"
Sep 09 20:42:07.554831 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:07.554787    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x" podStartSLOduration=2.480405106 podStartE2EDuration="4.554773175s" totalImagesPullingTime="2.074368069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:07.553393995 +0000 UTC m=+587.704066332" watchObservedRunningTime="2026-09-09 20:42:07.554773175 +0000 UTC m=+587.705445518"
Sep 09 20:42:07.869639 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:07.869554    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:07.916992 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:07.916963    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"]
Sep 09 20:42:07.925655 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:07.925628    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:07.929597 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:07.929576    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-h4wzg\""
Sep 09 20:42:07.943576 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:07.943549    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"]
Sep 09 20:42:08.008384 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.008348    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" (UID: \"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:08.008384 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.008386    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5chxg\" (UniqueName: \"kubernetes.io/projected/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-kube-api-access-5chxg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" (UID: \"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:08.091551 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.091517    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:08.109794 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.109770    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5chxg\" (UniqueName: \"kubernetes.io/projected/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-kube-api-access-5chxg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" (UID: \"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:08.109893 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.109849    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" (UID: \"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:08.110235 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.110219    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" (UID: \"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:08.121443 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.121379    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5chxg\" (UniqueName: \"kubernetes.io/projected/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-kube-api-access-5chxg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" (UID: \"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:08.235197 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.235152    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:08.261736 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.261700    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:08.378497 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.378400    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:08.515348 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.515318    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"]
Sep 09 20:42:08.518298 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:42:08.518271    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a7ace0_4567_4c6e_a0d0_21793b9bb9e9.slice/crio-3f93c87821f7429e255b3aefaa6773ddfdd430e5f07953d887af2e8ca6ff0a9c WatchSource:0}: Error finding container 3f93c87821f7429e255b3aefaa6773ddfdd430e5f07953d887af2e8ca6ff0a9c: Status 404 returned error can't find the container with id 3f93c87821f7429e255b3aefaa6773ddfdd430e5f07953d887af2e8ca6ff0a9c
Sep 09 20:42:08.530217 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:08.530189    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb" event={"ID":"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9","Type":"ContainerStarted","Data":"3f93c87821f7429e255b3aefaa6773ddfdd430e5f07953d887af2e8ca6ff0a9c"}
Sep 09 20:42:10.327104 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:10.327075    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-wgr2t"]
Sep 09 20:42:10.330568 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:10.330549    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-wgr2t"
Sep 09 20:42:10.332667 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:10.332649    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bnm8h\""
Sep 09 20:42:10.343294 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:10.343271    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-wgr2t"]
Sep 09 20:42:10.426021 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:10.425987    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxglk\" (UniqueName: \"kubernetes.io/projected/a5138784-3a4c-4e84-b161-bb8df8eaceef-kube-api-access-kxglk\") pod \"authorino-operator-76b478bd78-wgr2t\" (UID: \"a5138784-3a4c-4e84-b161-bb8df8eaceef\") " pod="kuadrant-system/authorino-operator-76b478bd78-wgr2t"
Sep 09 20:42:10.526528 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:10.526497    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxglk\" (UniqueName: \"kubernetes.io/projected/a5138784-3a4c-4e84-b161-bb8df8eaceef-kube-api-access-kxglk\") pod \"authorino-operator-76b478bd78-wgr2t\" (UID: \"a5138784-3a4c-4e84-b161-bb8df8eaceef\") " pod="kuadrant-system/authorino-operator-76b478bd78-wgr2t"
Sep 09 20:42:10.543730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:10.543703    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxglk\" (UniqueName: \"kubernetes.io/projected/a5138784-3a4c-4e84-b161-bb8df8eaceef-kube-api-access-kxglk\") pod \"authorino-operator-76b478bd78-wgr2t\" (UID: \"a5138784-3a4c-4e84-b161-bb8df8eaceef\") " pod="kuadrant-system/authorino-operator-76b478bd78-wgr2t"
Sep 09 20:42:10.643521 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:10.643449    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-wgr2t"
Sep 09 20:42:10.922190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:10.922141    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-wgr2t"]
Sep 09 20:42:10.924050 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:42:10.924017    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5138784_3a4c_4e84_b161_bb8df8eaceef.slice/crio-46a6bdeaa7bcadc59b2806d32717a7f0fb4e084b3fae2e3ff6db0382f5717078 WatchSource:0}: Error finding container 46a6bdeaa7bcadc59b2806d32717a7f0fb4e084b3fae2e3ff6db0382f5717078: Status 404 returned error can't find the container with id 46a6bdeaa7bcadc59b2806d32717a7f0fb4e084b3fae2e3ff6db0382f5717078
Sep 09 20:42:11.548264 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:11.548233    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-wgr2t" event={"ID":"a5138784-3a4c-4e84-b161-bb8df8eaceef","Type":"ContainerStarted","Data":"46a6bdeaa7bcadc59b2806d32717a7f0fb4e084b3fae2e3ff6db0382f5717078"}
Sep 09 20:42:11.636492 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:11.636458    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:11.737253 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:11.737216    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:12.556239 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:12.556202    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb" event={"ID":"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9","Type":"ContainerStarted","Data":"148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a"}
Sep 09 20:42:12.556640 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:12.556282    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:12.583446 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:12.580247    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb" podStartSLOduration=1.750943929 podStartE2EDuration="5.580227555s" totalImagesPullingTime="3.829283626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:12.578132367 +0000 UTC m=+592.728804712" watchObservedRunningTime="2026-09-09 20:42:12.580227555 +0000 UTC m=+592.730899899"
Sep 09 20:42:13.562250 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:13.562213    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-wgr2t" event={"ID":"a5138784-3a4c-4e84-b161-bb8df8eaceef","Type":"ContainerStarted","Data":"ec18a7ee59ddbe34cf3293afb88e05a14157d09b0c7f25abacea2cec2ff86e6d"}
Sep 09 20:42:13.579257 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:13.579201    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-wgr2t" podStartSLOduration=1.484679251 podStartE2EDuration="3.579183585s" totalImagesPullingTime="2.094504334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:13.578673646 +0000 UTC m=+593.729345990" watchObservedRunningTime="2026-09-09 20:42:13.579183585 +0000 UTC m=+593.729855921"
Sep 09 20:42:14.567207 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:14.567180    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-wgr2t"
Sep 09 20:42:14.633441 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:14.633404    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:14.698349 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:14.698323    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:15.136231 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:15.136190    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:15.438557 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:15.438473    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:16.520387 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:16.520359    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:18.533232 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:18.533202    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kjh5x"
Sep 09 20:42:19.643406 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:19.643371    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:19.743982 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:19.743950    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:19.844969 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:19.844935    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:20.260544 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:20.260516    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:42:20.260949 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:20.260929    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:42:23.564182 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:23.564132    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:24.698788 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.698749    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"]
Sep 09 20:42:24.703822 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.703800    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"
Sep 09 20:42:24.723764 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.723735    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"]
Sep 09 20:42:24.732093 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.732069    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pw6j\" (UniqueName: \"kubernetes.io/projected/848869cd-b098-4697-acfc-263fc5ce992f-kube-api-access-7pw6j\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-c959d\" (UID: \"848869cd-b098-4697-acfc-263fc5ce992f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"
Sep 09 20:42:24.732242 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.732129    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/848869cd-b098-4697-acfc-263fc5ce992f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-c959d\" (UID: \"848869cd-b098-4697-acfc-263fc5ce992f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"
Sep 09 20:42:24.801542 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.801520    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"]
Sep 09 20:42:24.801708 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:24.801692    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-7pw6j], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d" podUID="848869cd-b098-4697-acfc-263fc5ce992f"
Sep 09 20:42:24.832886 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.832865    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/848869cd-b098-4697-acfc-263fc5ce992f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-c959d\" (UID: \"848869cd-b098-4697-acfc-263fc5ce992f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"
Sep 09 20:42:24.832981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.832915    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pw6j\" (UniqueName: \"kubernetes.io/projected/848869cd-b098-4697-acfc-263fc5ce992f-kube-api-access-7pw6j\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-c959d\" (UID: \"848869cd-b098-4697-acfc-263fc5ce992f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"
Sep 09 20:42:24.833238 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.833218    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/848869cd-b098-4697-acfc-263fc5ce992f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-c959d\" (UID: \"848869cd-b098-4697-acfc-263fc5ce992f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"
Sep 09 20:42:24.851795 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.851769    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pw6j\" (UniqueName: \"kubernetes.io/projected/848869cd-b098-4697-acfc-263fc5ce992f-kube-api-access-7pw6j\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-c959d\" (UID: \"848869cd-b098-4697-acfc-263fc5ce992f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"
Sep 09 20:42:24.854640 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.854617    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"]
Sep 09 20:42:24.857946 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.857931    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:24.883545 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.883517    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"]
Sep 09 20:42:24.933781 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.933758    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2x66\" (UniqueName: \"kubernetes.io/projected/027b6363-500f-4f99-ae77-300ddb9287d4-kube-api-access-x2x66\") pod \"kuadrant-operator-controller-manager-5f685c496f-2x66t\" (UID: \"027b6363-500f-4f99-ae77-300ddb9287d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:24.933874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:24.933790    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/027b6363-500f-4f99-ae77-300ddb9287d4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-2x66t\" (UID: \"027b6363-500f-4f99-ae77-300ddb9287d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:25.034954 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.034916    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2x66\" (UniqueName: \"kubernetes.io/projected/027b6363-500f-4f99-ae77-300ddb9287d4-kube-api-access-x2x66\") pod \"kuadrant-operator-controller-manager-5f685c496f-2x66t\" (UID: \"027b6363-500f-4f99-ae77-300ddb9287d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:25.035064 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.034970    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/027b6363-500f-4f99-ae77-300ddb9287d4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-2x66t\" (UID: \"027b6363-500f-4f99-ae77-300ddb9287d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:25.035312 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.035297    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/027b6363-500f-4f99-ae77-300ddb9287d4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-2x66t\" (UID: \"027b6363-500f-4f99-ae77-300ddb9287d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:25.055875 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.055850    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2x66\" (UniqueName: \"kubernetes.io/projected/027b6363-500f-4f99-ae77-300ddb9287d4-kube-api-access-x2x66\") pod \"kuadrant-operator-controller-manager-5f685c496f-2x66t\" (UID: \"027b6363-500f-4f99-ae77-300ddb9287d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:25.167330 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.167308    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:25.449944 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.449922    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"]
Sep 09 20:42:25.452764 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:42:25.452741    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027b6363_500f_4f99_ae77_300ddb9287d4.slice/crio-9fc1e59542fb07d73370cd3b1f0c66f960589ec72ea03b17cdef2c42ba365d17 WatchSource:0}: Error finding container 9fc1e59542fb07d73370cd3b1f0c66f960589ec72ea03b17cdef2c42ba365d17: Status 404 returned error can't find the container with id 9fc1e59542fb07d73370cd3b1f0c66f960589ec72ea03b17cdef2c42ba365d17
Sep 09 20:42:25.574538 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.574511    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-wgr2t"
Sep 09 20:42:25.622932 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.622899    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t" event={"ID":"027b6363-500f-4f99-ae77-300ddb9287d4","Type":"ContainerStarted","Data":"d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557"}
Sep 09 20:42:25.622932 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.622933    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"
Sep 09 20:42:25.623186 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.622950    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t" event={"ID":"027b6363-500f-4f99-ae77-300ddb9287d4","Type":"ContainerStarted","Data":"9fc1e59542fb07d73370cd3b1f0c66f960589ec72ea03b17cdef2c42ba365d17"}
Sep 09 20:42:25.623186 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.622976    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:25.633216 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.633144    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"
Sep 09 20:42:25.647360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.647318    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t" podStartSLOduration=1.6473042869999999 podStartE2EDuration="1.647304287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:25.645639905 +0000 UTC m=+605.796312239" watchObservedRunningTime="2026-09-09 20:42:25.647304287 +0000 UTC m=+605.797976630"
Sep 09 20:42:25.723186 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.723146    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"]
Sep 09 20:42:25.739558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.739525    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pw6j\" (UniqueName: \"kubernetes.io/projected/848869cd-b098-4697-acfc-263fc5ce992f-kube-api-access-7pw6j\") pod \"848869cd-b098-4697-acfc-263fc5ce992f\" (UID: \"848869cd-b098-4697-acfc-263fc5ce992f\") "
Sep 09 20:42:25.739739 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.739617    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/848869cd-b098-4697-acfc-263fc5ce992f-extensions-socket-volume\") pod \"848869cd-b098-4697-acfc-263fc5ce992f\" (UID: \"848869cd-b098-4697-acfc-263fc5ce992f\") "
Sep 09 20:42:25.740138 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.740087    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/848869cd-b098-4697-acfc-263fc5ce992f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "848869cd-b098-4697-acfc-263fc5ce992f" (UID: "848869cd-b098-4697-acfc-263fc5ce992f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:42:25.741730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.741706    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/848869cd-b098-4697-acfc-263fc5ce992f-kube-api-access-7pw6j" (OuterVolumeSpecName: "kube-api-access-7pw6j") pod "848869cd-b098-4697-acfc-263fc5ce992f" (UID: "848869cd-b098-4697-acfc-263fc5ce992f"). InnerVolumeSpecName "kube-api-access-7pw6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:42:25.744556 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.744531    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"]
Sep 09 20:42:25.765789 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.765768    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"]
Sep 09 20:42:25.773438 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.773417    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"]
Sep 09 20:42:25.786979 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.786954    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"]
Sep 09 20:42:25.787401 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.787385    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027b6363-500f-4f99-ae77-300ddb9287d4" containerName="manager"
Sep 09 20:42:25.787456 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.787404    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="027b6363-500f-4f99-ae77-300ddb9287d4" containerName="manager"
Sep 09 20:42:25.787499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.787485    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="027b6363-500f-4f99-ae77-300ddb9287d4" containerName="manager"
Sep 09 20:42:25.790651 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.790635    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"]
Sep 09 20:42:25.790747 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.790733    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:25.790820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.790803    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" containerName="manager" containerID="cri-o://148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a" gracePeriod=2
Sep 09 20:42:25.794336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.794317    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"]
Sep 09 20:42:25.812095 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.812073    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"]
Sep 09 20:42:25.817157 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.817136    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"]
Sep 09 20:42:25.817342 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.817322    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx" podUID="53248ed7-d834-4bff-8746-aee79273e1f6" containerName="manager" containerID="cri-o://9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989" gracePeriod=2
Sep 09 20:42:25.826820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.826794    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"]
Sep 09 20:42:25.837987 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:25.837956    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:25.840348 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.840323    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/172cbbca-a8b4-4907-a026-bf9744e16d95-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-b4dtx\" (UID: \"172cbbca-a8b4-4907-a026-bf9744e16d95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:25.840445 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.840353    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kfn4\" (UniqueName: \"kubernetes.io/projected/172cbbca-a8b4-4907-a026-bf9744e16d95-kube-api-access-6kfn4\") pod \"kuadrant-operator-controller-manager-5f685c496f-b4dtx\" (UID: \"172cbbca-a8b4-4907-a026-bf9744e16d95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:25.840445 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.840408    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pw6j\" (UniqueName: \"kubernetes.io/projected/848869cd-b098-4697-acfc-263fc5ce992f-kube-api-access-7pw6j\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:25.840445 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.840419    2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/848869cd-b098-4697-acfc-263fc5ce992f-extensions-socket-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:25.856678 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:25.855083    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:25.857218 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:25.857192    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-k7lzx\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="53248ed7-d834-4bff-8746-aee79273e1f6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:25.857539 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.857519    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"]
Sep 09 20:42:25.857829 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.857814    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53248ed7-d834-4bff-8746-aee79273e1f6" containerName="manager"
Sep 09 20:42:25.857892 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.857832    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="53248ed7-d834-4bff-8746-aee79273e1f6" containerName="manager"
Sep 09 20:42:25.857892 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.857844    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" containerName="manager"
Sep 09 20:42:25.857892 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.857853    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" containerName="manager"
Sep 09 20:42:25.858028 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.857935    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" containerName="manager"
Sep 09 20:42:25.858028 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.857946    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="53248ed7-d834-4bff-8746-aee79273e1f6" containerName="manager"
Sep 09 20:42:25.860896 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.860877    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:25.863193 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:25.863151    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:25.864746 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:25.864725    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-k7lzx\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="53248ed7-d834-4bff-8746-aee79273e1f6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:25.869856 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.869836    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r"]
Sep 09 20:42:25.872714 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.872697    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r"
Sep 09 20:42:25.884339 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.884316    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"]
Sep 09 20:42:25.889454 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.889432    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r"]
Sep 09 20:42:25.902778 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:25.902753    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-k7lzx\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="53248ed7-d834-4bff-8746-aee79273e1f6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:25.931032 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:25.930996    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:25.941261 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.941229    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79psr\" (UniqueName: \"kubernetes.io/projected/f5076a56-0459-463a-a676-aba0f9fa56f2-kube-api-access-79psr\") pod \"limitador-operator-controller-manager-74495c69f8-5np4r\" (UID: \"f5076a56-0459-463a-a676-aba0f9fa56f2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r"
Sep 09 20:42:25.941364 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.941288    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4df9b10b-8f3d-43ff-9e81-958600d1275d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mggds\" (UID: \"4df9b10b-8f3d-43ff-9e81-958600d1275d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:25.941406 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.941359    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-427fh\" (UniqueName: \"kubernetes.io/projected/4df9b10b-8f3d-43ff-9e81-958600d1275d-kube-api-access-427fh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mggds\" (UID: \"4df9b10b-8f3d-43ff-9e81-958600d1275d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:25.941441 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.941422    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/172cbbca-a8b4-4907-a026-bf9744e16d95-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-b4dtx\" (UID: \"172cbbca-a8b4-4907-a026-bf9744e16d95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:25.941475 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.941451    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kfn4\" (UniqueName: \"kubernetes.io/projected/172cbbca-a8b4-4907-a026-bf9744e16d95-kube-api-access-6kfn4\") pod \"kuadrant-operator-controller-manager-5f685c496f-b4dtx\" (UID: \"172cbbca-a8b4-4907-a026-bf9744e16d95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:25.941850 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.941827    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/172cbbca-a8b4-4907-a026-bf9744e16d95-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-b4dtx\" (UID: \"172cbbca-a8b4-4907-a026-bf9744e16d95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:25.950258 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:25.950226    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kfn4\" (UniqueName: \"kubernetes.io/projected/172cbbca-a8b4-4907-a026-bf9744e16d95-kube-api-access-6kfn4\") pod \"kuadrant-operator-controller-manager-5f685c496f-b4dtx\" (UID: \"172cbbca-a8b4-4907-a026-bf9744e16d95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:26.042810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.042782    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79psr\" (UniqueName: \"kubernetes.io/projected/f5076a56-0459-463a-a676-aba0f9fa56f2-kube-api-access-79psr\") pod \"limitador-operator-controller-manager-74495c69f8-5np4r\" (UID: \"f5076a56-0459-463a-a676-aba0f9fa56f2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r"
Sep 09 20:42:26.042897 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.042824    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4df9b10b-8f3d-43ff-9e81-958600d1275d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mggds\" (UID: \"4df9b10b-8f3d-43ff-9e81-958600d1275d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:26.042897 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.042862    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-427fh\" (UniqueName: \"kubernetes.io/projected/4df9b10b-8f3d-43ff-9e81-958600d1275d-kube-api-access-427fh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mggds\" (UID: \"4df9b10b-8f3d-43ff-9e81-958600d1275d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:26.043332 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.043310    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4df9b10b-8f3d-43ff-9e81-958600d1275d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mggds\" (UID: \"4df9b10b-8f3d-43ff-9e81-958600d1275d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:26.054385 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.054364    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:26.056097 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.056078    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-427fh\" (UniqueName: \"kubernetes.io/projected/4df9b10b-8f3d-43ff-9e81-958600d1275d-kube-api-access-427fh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mggds\" (UID: \"4df9b10b-8f3d-43ff-9e81-958600d1275d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:26.056291 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.056274    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79psr\" (UniqueName: \"kubernetes.io/projected/f5076a56-0459-463a-a676-aba0f9fa56f2-kube-api-access-79psr\") pod \"limitador-operator-controller-manager-74495c69f8-5np4r\" (UID: \"f5076a56-0459-463a-a676-aba0f9fa56f2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r"
Sep 09 20:42:26.056497 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:26.056478    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-k7lzx\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="53248ed7-d834-4bff-8746-aee79273e1f6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:26.057208 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.057189    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:26.058312 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:26.058292    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:26.060796 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:26.060776    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9dgdb\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:26.063962 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:26.063943    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-k7lzx\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="53248ed7-d834-4bff-8746-aee79273e1f6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:26.102136 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.102115    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:26.144112 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.144084    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5chxg\" (UniqueName: \"kubernetes.io/projected/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-kube-api-access-5chxg\") pod \"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9\" (UID: \"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9\") "
Sep 09 20:42:26.144259 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.144132    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-extensions-socket-volume\") pod \"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9\" (UID: \"92a7ace0-4567-4c6e-a0d0-21793b9bb9e9\") "
Sep 09 20:42:26.144259 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.144218    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dff9j\" (UniqueName: \"kubernetes.io/projected/53248ed7-d834-4bff-8746-aee79273e1f6-kube-api-access-dff9j\") pod \"53248ed7-d834-4bff-8746-aee79273e1f6\" (UID: \"53248ed7-d834-4bff-8746-aee79273e1f6\") "
Sep 09 20:42:26.144534 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.144504    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" (UID: "92a7ace0-4567-4c6e-a0d0-21793b9bb9e9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:42:26.146196 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.146149    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-kube-api-access-5chxg" (OuterVolumeSpecName: "kube-api-access-5chxg") pod "92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" (UID: "92a7ace0-4567-4c6e-a0d0-21793b9bb9e9"). InnerVolumeSpecName "kube-api-access-5chxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:42:26.146295 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.146268    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53248ed7-d834-4bff-8746-aee79273e1f6-kube-api-access-dff9j" (OuterVolumeSpecName: "kube-api-access-dff9j") pod "53248ed7-d834-4bff-8746-aee79273e1f6" (UID: "53248ed7-d834-4bff-8746-aee79273e1f6"). InnerVolumeSpecName "kube-api-access-dff9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:42:26.210702 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.210674    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:26.216825 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.216804    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r"
Sep 09 20:42:26.245371 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.245342    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5chxg\" (UniqueName: \"kubernetes.io/projected/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-kube-api-access-5chxg\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:26.245496 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.245374    2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9-extensions-socket-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:26.245496 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.245389    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dff9j\" (UniqueName: \"kubernetes.io/projected/53248ed7-d834-4bff-8746-aee79273e1f6-kube-api-access-dff9j\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:26.283918 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.283892    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53248ed7-d834-4bff-8746-aee79273e1f6" path="/var/lib/kubelet/pods/53248ed7-d834-4bff-8746-aee79273e1f6/volumes"
Sep 09 20:42:26.284309 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.284295    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="848869cd-b098-4697-acfc-263fc5ce992f" path="/var/lib/kubelet/pods/848869cd-b098-4697-acfc-263fc5ce992f/volumes"
Sep 09 20:42:26.284595 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.284551    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" path="/var/lib/kubelet/pods/92a7ace0-4567-4c6e-a0d0-21793b9bb9e9/volumes"
Sep 09 20:42:26.394407 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.394385    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"]
Sep 09 20:42:26.396475 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:42:26.396452    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod172cbbca_a8b4_4907_a026_bf9744e16d95.slice/crio-d98692d004d8587b909ee6c68d731d2546f4f3ce866e12ae626bd714528775fe WatchSource:0}: Error finding container d98692d004d8587b909ee6c68d731d2546f4f3ce866e12ae626bd714528775fe: Status 404 returned error can't find the container with id d98692d004d8587b909ee6c68d731d2546f4f3ce866e12ae626bd714528775fe
Sep 09 20:42:26.496190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.496152    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r"]
Sep 09 20:42:26.497644 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:42:26.497322    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5076a56_0459_463a_a676_aba0f9fa56f2.slice/crio-ea840cd8e472024751c58f0ef5c743a724b7d907884fff602e27e9e9807ce354 WatchSource:0}: Error finding container ea840cd8e472024751c58f0ef5c743a724b7d907884fff602e27e9e9807ce354: Status 404 returned error can't find the container with id ea840cd8e472024751c58f0ef5c743a724b7d907884fff602e27e9e9807ce354
Sep 09 20:42:26.499488 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.499452    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"]
Sep 09 20:42:26.501565 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:42:26.501544    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df9b10b_8f3d_43ff_9e81_958600d1275d.slice/crio-f0c20192fa324d0b504154fdeebcee3eacf6b49aa874009c38ca94d3d09e3177 WatchSource:0}: Error finding container f0c20192fa324d0b504154fdeebcee3eacf6b49aa874009c38ca94d3d09e3177: Status 404 returned error can't find the container with id f0c20192fa324d0b504154fdeebcee3eacf6b49aa874009c38ca94d3d09e3177
Sep 09 20:42:26.629252 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.629223    2550 generic.go:356] "Generic (PLEG): container finished" podID="92a7ace0-4567-4c6e-a0d0-21793b9bb9e9" containerID="148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a" exitCode=0
Sep 09 20:42:26.629365 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.629267    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9dgdb"
Sep 09 20:42:26.629365 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.629321    2550 scope.go:117] "RemoveContainer" containerID="148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a"
Sep 09 20:42:26.631521 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.631469    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r" event={"ID":"f5076a56-0459-463a-a676-aba0f9fa56f2","Type":"ContainerStarted","Data":"788c37be784aca8420520eae53ffd309ce0ba729e4d17cf9745bc97475fcabd5"}
Sep 09 20:42:26.631625 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.631522    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r" event={"ID":"f5076a56-0459-463a-a676-aba0f9fa56f2","Type":"ContainerStarted","Data":"ea840cd8e472024751c58f0ef5c743a724b7d907884fff602e27e9e9807ce354"}
Sep 09 20:42:26.631947 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.631792    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r"
Sep 09 20:42:26.633948 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.633918    2550 generic.go:356] "Generic (PLEG): container finished" podID="53248ed7-d834-4bff-8746-aee79273e1f6" containerID="9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989" exitCode=0
Sep 09 20:42:26.633948 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.633952    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-k7lzx"
Sep 09 20:42:26.636817 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.636696    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds" event={"ID":"4df9b10b-8f3d-43ff-9e81-958600d1275d","Type":"ContainerStarted","Data":"4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b"}
Sep 09 20:42:26.636817 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.636746    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:26.636817 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.636788    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds" event={"ID":"4df9b10b-8f3d-43ff-9e81-958600d1275d","Type":"ContainerStarted","Data":"f0c20192fa324d0b504154fdeebcee3eacf6b49aa874009c38ca94d3d09e3177"}
Sep 09 20:42:26.639534 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.639513    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-c959d"
Sep 09 20:42:26.639534 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.639523    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx" event={"ID":"172cbbca-a8b4-4907-a026-bf9744e16d95","Type":"ContainerStarted","Data":"0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0"}
Sep 09 20:42:26.639669 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.639567    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx" event={"ID":"172cbbca-a8b4-4907-a026-bf9744e16d95","Type":"ContainerStarted","Data":"d98692d004d8587b909ee6c68d731d2546f4f3ce866e12ae626bd714528775fe"}
Sep 09 20:42:26.639812 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:26.639790    2550 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t" container="manager"
Sep 09 20:42:26.639880 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.639822    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:26.641864 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.641844    2550 scope.go:117] "RemoveContainer" containerID="148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a"
Sep 09 20:42:26.642130 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:26.642111    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a\": container with ID starting with 148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a not found: ID does not exist" containerID="148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a"
Sep 09 20:42:26.642240 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.642137    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a"} err="failed to get container status \"148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a\": rpc error: code = NotFound desc = could not find container \"148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a\": container with ID starting with 148c0cb2160e53ca89e9a40ec59bb8e65241602fafc9a79c805b23a87cd8556a not found: ID does not exist"
Sep 09 20:42:26.642240 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.642151    2550 scope.go:117] "RemoveContainer" containerID="9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989"
Sep 09 20:42:26.652117 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.652097    2550 scope.go:117] "RemoveContainer" containerID="9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989"
Sep 09 20:42:26.652598 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:26.652561    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989\": container with ID starting with 9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989 not found: ID does not exist" containerID="9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989"
Sep 09 20:42:26.652694 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.652599    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989"} err="failed to get container status \"9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989\": rpc error: code = NotFound desc = could not find container \"9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989\": container with ID starting with 9d8cd5db1f7a9aece24e742635d92ee512baa607dec47ae46dbfdb610d391989 not found: ID does not exist"
Sep 09 20:42:26.658573 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.658526    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r" podStartSLOduration=1.6585100270000002 podStartE2EDuration="1.658510027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:26.656003366 +0000 UTC m=+606.806675710" watchObservedRunningTime="2026-09-09 20:42:26.658510027 +0000 UTC m=+606.809182371"
Sep 09 20:42:26.678689 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.678650    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx" podStartSLOduration=1.678634131 podStartE2EDuration="1.678634131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:26.677240328 +0000 UTC m=+606.827912672" watchObservedRunningTime="2026-09-09 20:42:26.678634131 +0000 UTC m=+606.829306473"
Sep 09 20:42:26.697835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:26.697798    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds" podStartSLOduration=1.697785338 podStartE2EDuration="1.697785338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:26.696276682 +0000 UTC m=+606.846949048" watchObservedRunningTime="2026-09-09 20:42:26.697785338 +0000 UTC m=+606.848457681"
Sep 09 20:42:27.648047 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:27.648009    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t" podUID="027b6363-500f-4f99-ae77-300ddb9287d4" containerName="manager" containerID="cri-o://d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557" gracePeriod=2
Sep 09 20:42:27.888091 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:27.888068    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:27.890060 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:27.890033    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-2x66t\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="027b6363-500f-4f99-ae77-300ddb9287d4" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:27.964907 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:27.964882    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/027b6363-500f-4f99-ae77-300ddb9287d4-extensions-socket-volume\") pod \"027b6363-500f-4f99-ae77-300ddb9287d4\" (UID: \"027b6363-500f-4f99-ae77-300ddb9287d4\") "
Sep 09 20:42:27.965028 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:27.964951    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2x66\" (UniqueName: \"kubernetes.io/projected/027b6363-500f-4f99-ae77-300ddb9287d4-kube-api-access-x2x66\") pod \"027b6363-500f-4f99-ae77-300ddb9287d4\" (UID: \"027b6363-500f-4f99-ae77-300ddb9287d4\") "
Sep 09 20:42:27.965285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:27.965260    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/027b6363-500f-4f99-ae77-300ddb9287d4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "027b6363-500f-4f99-ae77-300ddb9287d4" (UID: "027b6363-500f-4f99-ae77-300ddb9287d4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:42:27.966886 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:27.966864    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/027b6363-500f-4f99-ae77-300ddb9287d4-kube-api-access-x2x66" (OuterVolumeSpecName: "kube-api-access-x2x66") pod "027b6363-500f-4f99-ae77-300ddb9287d4" (UID: "027b6363-500f-4f99-ae77-300ddb9287d4"). InnerVolumeSpecName "kube-api-access-x2x66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:42:28.065485 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:28.065446    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2x66\" (UniqueName: \"kubernetes.io/projected/027b6363-500f-4f99-ae77-300ddb9287d4-kube-api-access-x2x66\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:28.065485 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:28.065477    2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/027b6363-500f-4f99-ae77-300ddb9287d4-extensions-socket-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:28.279681 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:28.279605    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="027b6363-500f-4f99-ae77-300ddb9287d4" path="/var/lib/kubelet/pods/027b6363-500f-4f99-ae77-300ddb9287d4/volumes"
Sep 09 20:42:28.473010 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:28.472974    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:28.572594 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:28.572502    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:28.655853 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:28.655820    2550 generic.go:356] "Generic (PLEG): container finished" podID="027b6363-500f-4f99-ae77-300ddb9287d4" containerID="d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557" exitCode=0
Sep 09 20:42:28.656287 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:28.655865    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2x66t"
Sep 09 20:42:28.656287 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:28.655923    2550 scope.go:117] "RemoveContainer" containerID="d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557"
Sep 09 20:42:28.664269 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:28.664249    2550 scope.go:117] "RemoveContainer" containerID="d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557"
Sep 09 20:42:28.664549 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:28.664531    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557\": container with ID starting with d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557 not found: ID does not exist" containerID="d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557"
Sep 09 20:42:28.664620 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:28.664557    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557"} err="failed to get container status \"d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557\": rpc error: code = NotFound desc = could not find container \"d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557\": container with ID starting with d5deac987304872b038ebdc60d657c33cc786c71a12cdc029e68970790eb8557 not found: ID does not exist"
Sep 09 20:42:31.083381 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:31.083348    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:31.184369 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:31.184326    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:34.086135 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:34.086094    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:34.186806 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:34.186773    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:37.651675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:37.651644    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:37.652096 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:37.651717    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5np4r"
Sep 09 20:42:38.658919 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.658884    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:38.716533 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.716503    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"]
Sep 09 20:42:38.716750 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.716728    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds" podUID="4df9b10b-8f3d-43ff-9e81-958600d1275d" containerName="manager" containerID="cri-o://4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b" gracePeriod=10
Sep 09 20:42:38.895944 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.895776    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:38.975515 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.975493    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:38.976863 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.976842    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"]
Sep 09 20:42:38.977141 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.977128    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4df9b10b-8f3d-43ff-9e81-958600d1275d" containerName="manager"
Sep 09 20:42:38.977199 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.977142    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4df9b10b-8f3d-43ff-9e81-958600d1275d" containerName="manager"
Sep 09 20:42:38.977236 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.977203    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4df9b10b-8f3d-43ff-9e81-958600d1275d" containerName="manager"
Sep 09 20:42:38.980108 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.980093    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:42:38.999063 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:38.999042    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"]
Sep 09 20:42:39.019795 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.019771    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:39.056679 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.056655    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-427fh\" (UniqueName: \"kubernetes.io/projected/4df9b10b-8f3d-43ff-9e81-958600d1275d-kube-api-access-427fh\") pod \"4df9b10b-8f3d-43ff-9e81-958600d1275d\" (UID: \"4df9b10b-8f3d-43ff-9e81-958600d1275d\") "
Sep 09 20:42:39.056828 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.056739    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4df9b10b-8f3d-43ff-9e81-958600d1275d-extensions-socket-volume\") pod \"4df9b10b-8f3d-43ff-9e81-958600d1275d\" (UID: \"4df9b10b-8f3d-43ff-9e81-958600d1275d\") "
Sep 09 20:42:39.056880 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.056855    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8ch2r\" (UID: \"3d05d316-d3e3-40bb-888f-7d821ca1d2f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:42:39.056915 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.056885    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2dr2\" (UniqueName: \"kubernetes.io/projected/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-kube-api-access-z2dr2\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8ch2r\" (UID: \"3d05d316-d3e3-40bb-888f-7d821ca1d2f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:42:39.057130 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.057104    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4df9b10b-8f3d-43ff-9e81-958600d1275d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4df9b10b-8f3d-43ff-9e81-958600d1275d" (UID: "4df9b10b-8f3d-43ff-9e81-958600d1275d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:42:39.058686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.058667    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4df9b10b-8f3d-43ff-9e81-958600d1275d-kube-api-access-427fh" (OuterVolumeSpecName: "kube-api-access-427fh") pod "4df9b10b-8f3d-43ff-9e81-958600d1275d" (UID: "4df9b10b-8f3d-43ff-9e81-958600d1275d"). InnerVolumeSpecName "kube-api-access-427fh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:42:39.157906 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.157870    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2dr2\" (UniqueName: \"kubernetes.io/projected/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-kube-api-access-z2dr2\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8ch2r\" (UID: \"3d05d316-d3e3-40bb-888f-7d821ca1d2f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:42:39.158081 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.157954    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8ch2r\" (UID: \"3d05d316-d3e3-40bb-888f-7d821ca1d2f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:42:39.158081 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.158009    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-427fh\" (UniqueName: \"kubernetes.io/projected/4df9b10b-8f3d-43ff-9e81-958600d1275d-kube-api-access-427fh\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:39.158081 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.158026    2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4df9b10b-8f3d-43ff-9e81-958600d1275d-extensions-socket-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:39.158339 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.158319    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8ch2r\" (UID: \"3d05d316-d3e3-40bb-888f-7d821ca1d2f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:42:39.168066 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.168037    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2dr2\" (UniqueName: \"kubernetes.io/projected/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-kube-api-access-z2dr2\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8ch2r\" (UID: \"3d05d316-d3e3-40bb-888f-7d821ca1d2f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:42:39.288756 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.288668    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:42:39.563115 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.563088    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"]
Sep 09 20:42:39.564505 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:42:39.564483    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d05d316_d3e3_40bb_888f_7d821ca1d2f4.slice/crio-80cfacf5a5d580fbdd34139f2dc940d4bcb1a95ec279f222e84073502b158ec5 WatchSource:0}: Error finding container 80cfacf5a5d580fbdd34139f2dc940d4bcb1a95ec279f222e84073502b158ec5: Status 404 returned error can't find the container with id 80cfacf5a5d580fbdd34139f2dc940d4bcb1a95ec279f222e84073502b158ec5
Sep 09 20:42:39.713241 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.713210    2550 generic.go:356] "Generic (PLEG): container finished" podID="4df9b10b-8f3d-43ff-9e81-958600d1275d" containerID="4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b" exitCode=0
Sep 09 20:42:39.713591 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.713255    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"
Sep 09 20:42:39.713591 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.713305    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds" event={"ID":"4df9b10b-8f3d-43ff-9e81-958600d1275d","Type":"ContainerDied","Data":"4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b"}
Sep 09 20:42:39.713591 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.713358    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds" event={"ID":"4df9b10b-8f3d-43ff-9e81-958600d1275d","Type":"ContainerDied","Data":"f0c20192fa324d0b504154fdeebcee3eacf6b49aa874009c38ca94d3d09e3177"}
Sep 09 20:42:39.713591 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.713385    2550 scope.go:117] "RemoveContainer" containerID="4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b"
Sep 09 20:42:39.715334 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.715309    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r" event={"ID":"3d05d316-d3e3-40bb-888f-7d821ca1d2f4","Type":"ContainerStarted","Data":"485b0880caf3a286bda627eaf289ebb4b8bafdd8e3934899bd7c8ac2bb6737ec"}
Sep 09 20:42:39.715441 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.715356    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r" event={"ID":"3d05d316-d3e3-40bb-888f-7d821ca1d2f4","Type":"ContainerStarted","Data":"80cfacf5a5d580fbdd34139f2dc940d4bcb1a95ec279f222e84073502b158ec5"}
Sep 09 20:42:39.715441 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.715436    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:42:39.721911 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.721760    2550 scope.go:117] "RemoveContainer" containerID="4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b"
Sep 09 20:42:39.722044 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:39.722025    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b\": container with ID starting with 4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b not found: ID does not exist" containerID="4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b"
Sep 09 20:42:39.722084 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.722051    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b"} err="failed to get container status \"4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b\": rpc error: code = NotFound desc = could not find container \"4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b\": container with ID starting with 4372654b10d692dcf559063d02a481b2e9b9b8ff08eee46a0acad8567bca968b not found: ID does not exist"
Sep 09 20:42:39.737690 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.737648    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r" podStartSLOduration=1.737635453 podStartE2EDuration="1.737635453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:39.735284614 +0000 UTC m=+619.885956957" watchObservedRunningTime="2026-09-09 20:42:39.737635453 +0000 UTC m=+619.888307795"
Sep 09 20:42:39.751564 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.751542    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"]
Sep 09 20:42:39.758916 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.758893    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mggds"]
Sep 09 20:42:39.871735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.871668    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:39.972329 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:39.972303    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:40.279527 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:40.279497    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4df9b10b-8f3d-43ff-9e81-958600d1275d" path="/var/lib/kubelet/pods/4df9b10b-8f3d-43ff-9e81-958600d1275d/volumes"
Sep 09 20:42:40.661936 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:40.661848    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:40.771539 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:40.771510    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:47.675515 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:47.675428    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:47.776244 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:47.776207    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:50.723356 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:50.723321    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:42:50.775284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:50.775255    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"]
Sep 09 20:42:50.775595 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:50.775529    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx" podUID="172cbbca-a8b4-4907-a026-bf9744e16d95" containerName="manager" containerID="cri-o://0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0" gracePeriod=10
Sep 09 20:42:51.017103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.017081    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:51.154814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.154782    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kfn4\" (UniqueName: \"kubernetes.io/projected/172cbbca-a8b4-4907-a026-bf9744e16d95-kube-api-access-6kfn4\") pod \"172cbbca-a8b4-4907-a026-bf9744e16d95\" (UID: \"172cbbca-a8b4-4907-a026-bf9744e16d95\") "
Sep 09 20:42:51.154968 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.154843    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/172cbbca-a8b4-4907-a026-bf9744e16d95-extensions-socket-volume\") pod \"172cbbca-a8b4-4907-a026-bf9744e16d95\" (UID: \"172cbbca-a8b4-4907-a026-bf9744e16d95\") "
Sep 09 20:42:51.155276 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.155254    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/172cbbca-a8b4-4907-a026-bf9744e16d95-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "172cbbca-a8b4-4907-a026-bf9744e16d95" (UID: "172cbbca-a8b4-4907-a026-bf9744e16d95"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:42:51.156841 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.156823    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/172cbbca-a8b4-4907-a026-bf9744e16d95-kube-api-access-6kfn4" (OuterVolumeSpecName: "kube-api-access-6kfn4") pod "172cbbca-a8b4-4907-a026-bf9744e16d95" (UID: "172cbbca-a8b4-4907-a026-bf9744e16d95"). InnerVolumeSpecName "kube-api-access-6kfn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:42:51.255987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.255901    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kfn4\" (UniqueName: \"kubernetes.io/projected/172cbbca-a8b4-4907-a026-bf9744e16d95-kube-api-access-6kfn4\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:51.255987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.255933    2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/172cbbca-a8b4-4907-a026-bf9744e16d95-extensions-socket-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:42:51.778688 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.778657    2550 generic.go:356] "Generic (PLEG): container finished" podID="172cbbca-a8b4-4907-a026-bf9744e16d95" containerID="0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0" exitCode=0
Sep 09 20:42:51.779085 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.778702    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"
Sep 09 20:42:51.779085 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.778720    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx" event={"ID":"172cbbca-a8b4-4907-a026-bf9744e16d95","Type":"ContainerDied","Data":"0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0"}
Sep 09 20:42:51.779085 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.778769    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx" event={"ID":"172cbbca-a8b4-4907-a026-bf9744e16d95","Type":"ContainerDied","Data":"d98692d004d8587b909ee6c68d731d2546f4f3ce866e12ae626bd714528775fe"}
Sep 09 20:42:51.779085 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.778793    2550 scope.go:117] "RemoveContainer" containerID="0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0"
Sep 09 20:42:51.787606 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.787586    2550 scope.go:117] "RemoveContainer" containerID="0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0"
Sep 09 20:42:51.787866 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:42:51.787847    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0\": container with ID starting with 0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0 not found: ID does not exist" containerID="0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0"
Sep 09 20:42:51.787918 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.787874    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0"} err="failed to get container status \"0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0\": rpc error: code = NotFound desc = could not find container \"0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0\": container with ID starting with 0aed6854279b4edd4f2ea66bd8a217b027da4aa754499c43ec55d8be3a7f2da0 not found: ID does not exist"
Sep 09 20:42:51.800873 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.800847    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"]
Sep 09 20:42:51.804975 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:51.804953    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-b4dtx"]
Sep 09 20:42:52.279000 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:52.278968    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="172cbbca-a8b4-4907-a026-bf9744e16d95" path="/var/lib/kubelet/pods/172cbbca-a8b4-4907-a026-bf9744e16d95/volumes"
Sep 09 20:42:54.208703 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:54.208666    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:54.308541 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:54.308505    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:55.107925 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.107892    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"]
Sep 09 20:42:55.108284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.108266    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="172cbbca-a8b4-4907-a026-bf9744e16d95" containerName="manager"
Sep 09 20:42:55.108284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.108285    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="172cbbca-a8b4-4907-a026-bf9744e16d95" containerName="manager"
Sep 09 20:42:55.108433 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.108375    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="172cbbca-a8b4-4907-a026-bf9744e16d95" containerName="manager"
Sep 09 20:42:55.110279 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.110255    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.112401 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.112382    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-95sfc\""
Sep 09 20:42:55.123282 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.123260    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"]
Sep 09 20:42:55.141495 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.141472    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:55.282979 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.282951    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.283368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.282985    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.283368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.283014    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.283368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.283085    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.283368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.283108    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.283368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.283134    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.283368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.283151    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4nmv\" (UniqueName: \"kubernetes.io/projected/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-kube-api-access-f4nmv\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.283368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.283190    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.283368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.283219    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.320838 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.320808    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:55.383981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.383917    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.383981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.383954    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.383981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.383979    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.384204 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.383995    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4nmv\" (UniqueName: \"kubernetes.io/projected/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-kube-api-access-f4nmv\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.384204 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.384042    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.384204 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.384096    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.384204 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.384133    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.384204 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.384160    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.384480 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.384230    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.384480 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.384400    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.384660 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.384636    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.384928 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.384906    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.385050 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.384919    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.385112 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.385081    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.386742 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.386722    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.386854 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.386794    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.392311 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.392292    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4nmv\" (UniqueName: \"kubernetes.io/projected/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-kube-api-access-f4nmv\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.392610 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.392589    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jgz67\" (UID: \"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.419579 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.419559    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:55.690874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.690847    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"]
Sep 09 20:42:55.692002 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:42:55.691980    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2ed2bd_5842_44e8_a0dc_4c81cf21ce9c.slice/crio-dcbcd35ebc73ab24e6115c42afa03c9cf9222b1d48f4c16555ef986f486ee69a WatchSource:0}: Error finding container dcbcd35ebc73ab24e6115c42afa03c9cf9222b1d48f4c16555ef986f486ee69a: Status 404 returned error can't find the container with id dcbcd35ebc73ab24e6115c42afa03c9cf9222b1d48f4c16555ef986f486ee69a
Sep 09 20:42:55.694321 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.694292    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:42:55.694398 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.694354    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:42:55.694398 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.694384    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:42:55.803564 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.803529    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" event={"ID":"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c","Type":"ContainerStarted","Data":"0ec38e1c5fcefeb840fac81706b7acba1bd3e1e459dc529b15ff8cbedfd7e4dd"}
Sep 09 20:42:55.803745 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.803580    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" event={"ID":"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c","Type":"ContainerStarted","Data":"dcbcd35ebc73ab24e6115c42afa03c9cf9222b1d48f4c16555ef986f486ee69a"}
Sep 09 20:42:55.822674 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:55.822625    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" podStartSLOduration=0.822610382 podStartE2EDuration="822.610382ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:55.820860581 +0000 UTC m=+635.971532924" watchObservedRunningTime="2026-09-09 20:42:55.822610382 +0000 UTC m=+635.973282725"
Sep 09 20:42:56.419723 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:56.419693    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:56.424289 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:56.424262    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:56.808807 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:56.808772    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:42:56.809794 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:42:56.809777    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:43:08.190586 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:08.190538    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:08.297155 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:08.297123    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:08.396424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:08.396379    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:09.021881 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.021851    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j99fj"]
Sep 09 20:43:09.026503 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.026484    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj"
Sep 09 20:43:09.028504 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.028482    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8rvsv\""
Sep 09 20:43:09.032009 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.031989    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j99fj"]
Sep 09 20:43:09.059667 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.059637    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:09.154769 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.154741    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:09.155205 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.155186    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-xsq4v"]
Sep 09 20:43:09.158314 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.158300    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xsq4v"
Sep 09 20:43:09.163482 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.163459    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xsq4v"]
Sep 09 20:43:09.188058 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.188036    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvvzg\" (UniqueName: \"kubernetes.io/projected/5d70c3a9-9c53-490b-a366-8a835251a4e4-kube-api-access-rvvzg\") pod \"authorino-7bcfd8f9cd-j99fj\" (UID: \"5d70c3a9-9c53-490b-a366-8a835251a4e4\") " pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj"
Sep 09 20:43:09.289208 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.289121    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m52w\" (UniqueName: \"kubernetes.io/projected/fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08-kube-api-access-5m52w\") pod \"authorino-59c974bd48-xsq4v\" (UID: \"fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08\") " pod="kuadrant-system/authorino-59c974bd48-xsq4v"
Sep 09 20:43:09.289208 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.289154    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvvzg\" (UniqueName: \"kubernetes.io/projected/5d70c3a9-9c53-490b-a366-8a835251a4e4-kube-api-access-rvvzg\") pod \"authorino-7bcfd8f9cd-j99fj\" (UID: \"5d70c3a9-9c53-490b-a366-8a835251a4e4\") " pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj"
Sep 09 20:43:09.297079 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.297051    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvvzg\" (UniqueName: \"kubernetes.io/projected/5d70c3a9-9c53-490b-a366-8a835251a4e4-kube-api-access-rvvzg\") pod \"authorino-7bcfd8f9cd-j99fj\" (UID: \"5d70c3a9-9c53-490b-a366-8a835251a4e4\") " pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj"
Sep 09 20:43:09.335905 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.335883    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj"
Sep 09 20:43:09.389888 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.389849    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m52w\" (UniqueName: \"kubernetes.io/projected/fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08-kube-api-access-5m52w\") pod \"authorino-59c974bd48-xsq4v\" (UID: \"fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08\") " pod="kuadrant-system/authorino-59c974bd48-xsq4v"
Sep 09 20:43:09.401088 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.401061    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m52w\" (UniqueName: \"kubernetes.io/projected/fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08-kube-api-access-5m52w\") pod \"authorino-59c974bd48-xsq4v\" (UID: \"fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08\") " pod="kuadrant-system/authorino-59c974bd48-xsq4v"
Sep 09 20:43:09.467792 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.467758    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xsq4v"
Sep 09 20:43:09.605386 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.605312    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j99fj"]
Sep 09 20:43:09.607923 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:43:09.607898    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d70c3a9_9c53_490b_a366_8a835251a4e4.slice/crio-b9be09b4689ca59e61d905d4889b241b5145d40b3ec20b03ed6ab8d1cfb84295 WatchSource:0}: Error finding container b9be09b4689ca59e61d905d4889b241b5145d40b3ec20b03ed6ab8d1cfb84295: Status 404 returned error can't find the container with id b9be09b4689ca59e61d905d4889b241b5145d40b3ec20b03ed6ab8d1cfb84295
Sep 09 20:43:09.735426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.735309    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xsq4v"]
Sep 09 20:43:09.737271 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:43:09.737250    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa8d1a95_7ed3_4fe7_adc1_d3a8df569f08.slice/crio-57d69cf7508d9b83487cf9551522d1c019ae42a895d7f6e40a5bbcb60b87b27a WatchSource:0}: Error finding container 57d69cf7508d9b83487cf9551522d1c019ae42a895d7f6e40a5bbcb60b87b27a: Status 404 returned error can't find the container with id 57d69cf7508d9b83487cf9551522d1c019ae42a895d7f6e40a5bbcb60b87b27a
Sep 09 20:43:09.873246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.873149    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xsq4v" event={"ID":"fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08","Type":"ContainerStarted","Data":"57d69cf7508d9b83487cf9551522d1c019ae42a895d7f6e40a5bbcb60b87b27a"}
Sep 09 20:43:09.874286 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:09.874266    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj" event={"ID":"5d70c3a9-9c53-490b-a366-8a835251a4e4","Type":"ContainerStarted","Data":"b9be09b4689ca59e61d905d4889b241b5145d40b3ec20b03ed6ab8d1cfb84295"}
Sep 09 20:43:11.887351 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:11.887263    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj" event={"ID":"5d70c3a9-9c53-490b-a366-8a835251a4e4","Type":"ContainerStarted","Data":"a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d"}
Sep 09 20:43:11.888730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:11.888702    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xsq4v" event={"ID":"fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08","Type":"ContainerStarted","Data":"27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa"}
Sep 09 20:43:11.900940 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:11.900896    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj" podStartSLOduration=0.929973693 podStartE2EDuration="2.900883653s" totalImagesPullingTime="1.97090996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:43:11.899364583 +0000 UTC m=+652.050036925" watchObservedRunningTime="2026-09-09 20:43:11.900883653 +0000 UTC m=+652.051555995"
Sep 09 20:43:11.914566 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:11.914526    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-xsq4v" podStartSLOduration=1.077562915 podStartE2EDuration="2.914513699s" totalImagesPullingTime="1.836950784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:43:11.912250289 +0000 UTC m=+652.062922642" watchObservedRunningTime="2026-09-09 20:43:11.914513699 +0000 UTC m=+652.065186041"
Sep 09 20:43:11.935409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:11.935383    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j99fj"]
Sep 09 20:43:13.899264 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:13.899221    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj" podUID="5d70c3a9-9c53-490b-a366-8a835251a4e4" containerName="authorino" containerID="cri-o://a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d" gracePeriod=30
Sep 09 20:43:14.148976 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.148956    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj"
Sep 09 20:43:14.225990 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.225965    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvvzg\" (UniqueName: \"kubernetes.io/projected/5d70c3a9-9c53-490b-a366-8a835251a4e4-kube-api-access-rvvzg\") pod \"5d70c3a9-9c53-490b-a366-8a835251a4e4\" (UID: \"5d70c3a9-9c53-490b-a366-8a835251a4e4\") "
Sep 09 20:43:14.227870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.227848    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d70c3a9-9c53-490b-a366-8a835251a4e4-kube-api-access-rvvzg" (OuterVolumeSpecName: "kube-api-access-rvvzg") pod "5d70c3a9-9c53-490b-a366-8a835251a4e4" (UID: "5d70c3a9-9c53-490b-a366-8a835251a4e4"). InnerVolumeSpecName "kube-api-access-rvvzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:43:14.326743 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.326718    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvvzg\" (UniqueName: \"kubernetes.io/projected/5d70c3a9-9c53-490b-a366-8a835251a4e4-kube-api-access-rvvzg\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:43:14.904978 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.904950    2550 generic.go:356] "Generic (PLEG): container finished" podID="5d70c3a9-9c53-490b-a366-8a835251a4e4" containerID="a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d" exitCode=0
Sep 09 20:43:14.905362 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.904988    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj"
Sep 09 20:43:14.905362 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.905012    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj" event={"ID":"5d70c3a9-9c53-490b-a366-8a835251a4e4","Type":"ContainerDied","Data":"a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d"}
Sep 09 20:43:14.905362 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.905041    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j99fj" event={"ID":"5d70c3a9-9c53-490b-a366-8a835251a4e4","Type":"ContainerDied","Data":"b9be09b4689ca59e61d905d4889b241b5145d40b3ec20b03ed6ab8d1cfb84295"}
Sep 09 20:43:14.905362 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.905057    2550 scope.go:117] "RemoveContainer" containerID="a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d"
Sep 09 20:43:14.913139 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.913124    2550 scope.go:117] "RemoveContainer" containerID="a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d"
Sep 09 20:43:14.913412 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:43:14.913392    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d\": container with ID starting with a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d not found: ID does not exist" containerID="a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d"
Sep 09 20:43:14.913479 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.913420    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d"} err="failed to get container status \"a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d\": rpc error: code = NotFound desc = could not find container \"a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d\": container with ID starting with a0da3f1f481cf80c23db9039e2278c36d97028aba8654a345d462badb156b52d not found: ID does not exist"
Sep 09 20:43:14.918872 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.918851    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j99fj"]
Sep 09 20:43:14.920757 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:14.920738    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j99fj"]
Sep 09 20:43:16.278987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:16.278943    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d70c3a9-9c53-490b-a366-8a835251a4e4" path="/var/lib/kubelet/pods/5d70c3a9-9c53-490b-a366-8a835251a4e4/volumes"
Sep 09 20:43:20.928146 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:20.928107    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:26.917057 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:26.917025    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:26.975422 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:26.975385    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:28.212510 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:28.212475    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:28.313882 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:28.313842    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:28.650550 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:28.650460    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:38.673413 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:38.673382    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:40.776535 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:40.776494    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:41.017052 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.017015    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-5bwgc"]
Sep 09 20:43:41.017340 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.017327    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d70c3a9-9c53-490b-a366-8a835251a4e4" containerName="authorino"
Sep 09 20:43:41.017389 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.017342    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d70c3a9-9c53-490b-a366-8a835251a4e4" containerName="authorino"
Sep 09 20:43:41.017423 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.017399    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d70c3a9-9c53-490b-a366-8a835251a4e4" containerName="authorino"
Sep 09 20:43:41.021813 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.021793    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5bwgc"
Sep 09 20:43:41.025467 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.025443    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-5bwgc"]
Sep 09 20:43:41.042716 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.042689    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:41.096185 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.096139    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-5bwgc"]
Sep 09 20:43:41.096380 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:43:41.096359    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2dngn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-5bwgc" podUID="df62eda8-9e07-4736-b595-029ef498b841"
Sep 09 20:43:41.117513 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.117483    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dngn\" (UniqueName: \"kubernetes.io/projected/df62eda8-9e07-4736-b595-029ef498b841-kube-api-access-2dngn\") pod \"authorino-557f876f69-5bwgc\" (UID: \"df62eda8-9e07-4736-b595-029ef498b841\") " pod="kuadrant-system/authorino-557f876f69-5bwgc"
Sep 09 20:43:41.124412 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.124388    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-psgfm"]
Sep 09 20:43:41.127622 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.127602    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-psgfm"
Sep 09 20:43:41.129584 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.129568    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 09 20:43:41.136145 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.136123    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-psgfm"]
Sep 09 20:43:41.218830 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.218797    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c2c81d82-0265-490b-bc53-c21ef8eeed21-tls-cert\") pod \"authorino-6d4dfc649f-psgfm\" (UID: \"c2c81d82-0265-490b-bc53-c21ef8eeed21\") " pod="kuadrant-system/authorino-6d4dfc649f-psgfm"
Sep 09 20:43:41.218986 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.218843    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dngn\" (UniqueName: \"kubernetes.io/projected/df62eda8-9e07-4736-b595-029ef498b841-kube-api-access-2dngn\") pod \"authorino-557f876f69-5bwgc\" (UID: \"df62eda8-9e07-4736-b595-029ef498b841\") " pod="kuadrant-system/authorino-557f876f69-5bwgc"
Sep 09 20:43:41.218986 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.218934    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c72jg\" (UniqueName: \"kubernetes.io/projected/c2c81d82-0265-490b-bc53-c21ef8eeed21-kube-api-access-c72jg\") pod \"authorino-6d4dfc649f-psgfm\" (UID: \"c2c81d82-0265-490b-bc53-c21ef8eeed21\") " pod="kuadrant-system/authorino-6d4dfc649f-psgfm"
Sep 09 20:43:41.236247 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.236221    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dngn\" (UniqueName: \"kubernetes.io/projected/df62eda8-9e07-4736-b595-029ef498b841-kube-api-access-2dngn\") pod \"authorino-557f876f69-5bwgc\" (UID: \"df62eda8-9e07-4736-b595-029ef498b841\") " pod="kuadrant-system/authorino-557f876f69-5bwgc"
Sep 09 20:43:41.283924 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.283845    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-psgfm"]
Sep 09 20:43:41.284123 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:43:41.284103    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-c72jg tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-psgfm" podUID="c2c81d82-0265-490b-bc53-c21ef8eeed21"
Sep 09 20:43:41.320111 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.320061    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c2c81d82-0265-490b-bc53-c21ef8eeed21-tls-cert\") pod \"authorino-6d4dfc649f-psgfm\" (UID: \"c2c81d82-0265-490b-bc53-c21ef8eeed21\") " pod="kuadrant-system/authorino-6d4dfc649f-psgfm"
Sep 09 20:43:41.320288 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.320185    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c72jg\" (UniqueName: \"kubernetes.io/projected/c2c81d82-0265-490b-bc53-c21ef8eeed21-kube-api-access-c72jg\") pod \"authorino-6d4dfc649f-psgfm\" (UID: \"c2c81d82-0265-490b-bc53-c21ef8eeed21\") " pod="kuadrant-system/authorino-6d4dfc649f-psgfm"
Sep 09 20:43:41.322594 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.322569    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c2c81d82-0265-490b-bc53-c21ef8eeed21-tls-cert\") pod \"authorino-6d4dfc649f-psgfm\" (UID: \"c2c81d82-0265-490b-bc53-c21ef8eeed21\") " pod="kuadrant-system/authorino-6d4dfc649f-psgfm"
Sep 09 20:43:41.324529 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.324506    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-759c79f5db-cqm8p"]
Sep 09 20:43:41.327770 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.327748    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-759c79f5db-cqm8p"
Sep 09 20:43:41.339116 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.339092    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-759c79f5db-cqm8p"]
Sep 09 20:43:41.340570 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.340547    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c72jg\" (UniqueName: \"kubernetes.io/projected/c2c81d82-0265-490b-bc53-c21ef8eeed21-kube-api-access-c72jg\") pod \"authorino-6d4dfc649f-psgfm\" (UID: \"c2c81d82-0265-490b-bc53-c21ef8eeed21\") " pod="kuadrant-system/authorino-6d4dfc649f-psgfm"
Sep 09 20:43:41.421004 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.420969    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2xp2\" (UniqueName: \"kubernetes.io/projected/baa32b51-d591-495e-b2a4-4eff0c55d850-kube-api-access-d2xp2\") pod \"authorino-759c79f5db-cqm8p\" (UID: \"baa32b51-d591-495e-b2a4-4eff0c55d850\") " pod="kuadrant-system/authorino-759c79f5db-cqm8p"
Sep 09 20:43:41.421191 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.421037    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/baa32b51-d591-495e-b2a4-4eff0c55d850-tls-cert\") pod \"authorino-759c79f5db-cqm8p\" (UID: \"baa32b51-d591-495e-b2a4-4eff0c55d850\") " pod="kuadrant-system/authorino-759c79f5db-cqm8p"
Sep 09 20:43:41.522183 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.522132    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/baa32b51-d591-495e-b2a4-4eff0c55d850-tls-cert\") pod \"authorino-759c79f5db-cqm8p\" (UID: \"baa32b51-d591-495e-b2a4-4eff0c55d850\") " pod="kuadrant-system/authorino-759c79f5db-cqm8p"
Sep 09 20:43:41.522368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.522237    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2xp2\" (UniqueName: \"kubernetes.io/projected/baa32b51-d591-495e-b2a4-4eff0c55d850-kube-api-access-d2xp2\") pod \"authorino-759c79f5db-cqm8p\" (UID: \"baa32b51-d591-495e-b2a4-4eff0c55d850\") " pod="kuadrant-system/authorino-759c79f5db-cqm8p"
Sep 09 20:43:41.524571 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.524546    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/baa32b51-d591-495e-b2a4-4eff0c55d850-tls-cert\") pod \"authorino-759c79f5db-cqm8p\" (UID: \"baa32b51-d591-495e-b2a4-4eff0c55d850\") " pod="kuadrant-system/authorino-759c79f5db-cqm8p"
Sep 09 20:43:41.530046 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.530024    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2xp2\" (UniqueName: \"kubernetes.io/projected/baa32b51-d591-495e-b2a4-4eff0c55d850-kube-api-access-d2xp2\") pod \"authorino-759c79f5db-cqm8p\" (UID: \"baa32b51-d591-495e-b2a4-4eff0c55d850\") " pod="kuadrant-system/authorino-759c79f5db-cqm8p"
Sep 09 20:43:41.637599 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.637537    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-759c79f5db-cqm8p"
Sep 09 20:43:41.905780 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:41.905692    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-759c79f5db-cqm8p"]
Sep 09 20:43:41.908378 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:43:41.908352    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbaa32b51_d591_495e_b2a4_4eff0c55d850.slice/crio-aa767e4d4b1b6d27897a5cf26e8aa8d0deb3497c25e8d762235db94c3be7abd2 WatchSource:0}: Error finding container aa767e4d4b1b6d27897a5cf26e8aa8d0deb3497c25e8d762235db94c3be7abd2: Status 404 returned error can't find the container with id aa767e4d4b1b6d27897a5cf26e8aa8d0deb3497c25e8d762235db94c3be7abd2
Sep 09 20:43:42.038262 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.038235    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-psgfm"
Sep 09 20:43:42.038420 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.038258    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-759c79f5db-cqm8p" event={"ID":"baa32b51-d591-495e-b2a4-4eff0c55d850","Type":"ContainerStarted","Data":"aa767e4d4b1b6d27897a5cf26e8aa8d0deb3497c25e8d762235db94c3be7abd2"}
Sep 09 20:43:42.038420 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.038363    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5bwgc"
Sep 09 20:43:42.046426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.046407    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-psgfm"
Sep 09 20:43:42.048409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.048395    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5bwgc"
Sep 09 20:43:42.127297 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.127266    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c2c81d82-0265-490b-bc53-c21ef8eeed21-tls-cert\") pod \"c2c81d82-0265-490b-bc53-c21ef8eeed21\" (UID: \"c2c81d82-0265-490b-bc53-c21ef8eeed21\") "
Sep 09 20:43:42.127439 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.127315    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dngn\" (UniqueName: \"kubernetes.io/projected/df62eda8-9e07-4736-b595-029ef498b841-kube-api-access-2dngn\") pod \"df62eda8-9e07-4736-b595-029ef498b841\" (UID: \"df62eda8-9e07-4736-b595-029ef498b841\") "
Sep 09 20:43:42.127439 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.127360    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c72jg\" (UniqueName: \"kubernetes.io/projected/c2c81d82-0265-490b-bc53-c21ef8eeed21-kube-api-access-c72jg\") pod \"c2c81d82-0265-490b-bc53-c21ef8eeed21\" (UID: \"c2c81d82-0265-490b-bc53-c21ef8eeed21\") "
Sep 09 20:43:42.129329 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.129307    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2c81d82-0265-490b-bc53-c21ef8eeed21-kube-api-access-c72jg" (OuterVolumeSpecName: "kube-api-access-c72jg") pod "c2c81d82-0265-490b-bc53-c21ef8eeed21" (UID: "c2c81d82-0265-490b-bc53-c21ef8eeed21"). InnerVolumeSpecName "kube-api-access-c72jg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:43:42.129420 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.129380    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df62eda8-9e07-4736-b595-029ef498b841-kube-api-access-2dngn" (OuterVolumeSpecName: "kube-api-access-2dngn") pod "df62eda8-9e07-4736-b595-029ef498b841" (UID: "df62eda8-9e07-4736-b595-029ef498b841"). InnerVolumeSpecName "kube-api-access-2dngn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:43:42.129420 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.129399    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2c81d82-0265-490b-bc53-c21ef8eeed21-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c2c81d82-0265-490b-bc53-c21ef8eeed21" (UID: "c2c81d82-0265-490b-bc53-c21ef8eeed21"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:43:42.228095 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.228069    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dngn\" (UniqueName: \"kubernetes.io/projected/df62eda8-9e07-4736-b595-029ef498b841-kube-api-access-2dngn\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:43:42.228095 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.228092    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c72jg\" (UniqueName: \"kubernetes.io/projected/c2c81d82-0265-490b-bc53-c21ef8eeed21-kube-api-access-c72jg\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:43:42.228287 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:42.228101    2550 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c2c81d82-0265-490b-bc53-c21ef8eeed21-tls-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:43:43.049948 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.049907    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-759c79f5db-cqm8p" event={"ID":"baa32b51-d591-495e-b2a4-4eff0c55d850","Type":"ContainerStarted","Data":"29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8"}
Sep 09 20:43:43.049948 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.049949    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5bwgc"
Sep 09 20:43:43.050497 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.050227    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-psgfm"
Sep 09 20:43:43.069774 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.069716    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-759c79f5db-cqm8p" podStartSLOduration=1.298390428 podStartE2EDuration="2.069697321s" totalImagesPullingTime="771.306893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:43:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:43:43.067862958 +0000 UTC m=+683.218535304" watchObservedRunningTime="2026-09-09 20:43:43.069697321 +0000 UTC m=+683.220369664"
Sep 09 20:43:43.103393 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.103359    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-5bwgc"]
Sep 09 20:43:43.105672 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.105645    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xsq4v"]
Sep 09 20:43:43.105831 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.105808    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-xsq4v" podUID="fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08" containerName="authorino" containerID="cri-o://27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa" gracePeriod=30
Sep 09 20:43:43.107729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.107700    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-5bwgc"]
Sep 09 20:43:43.136785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.136764    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-psgfm"]
Sep 09 20:43:43.141097 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.141077    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-psgfm"]
Sep 09 20:43:43.340544 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.340519    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xsq4v"
Sep 09 20:43:43.437717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.437685    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m52w\" (UniqueName: \"kubernetes.io/projected/fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08-kube-api-access-5m52w\") pod \"fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08\" (UID: \"fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08\") "
Sep 09 20:43:43.439675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.439646    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08-kube-api-access-5m52w" (OuterVolumeSpecName: "kube-api-access-5m52w") pod "fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08" (UID: "fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08"). InnerVolumeSpecName "kube-api-access-5m52w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:43:43.538315 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:43.538289    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m52w\" (UniqueName: \"kubernetes.io/projected/fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08-kube-api-access-5m52w\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:43:44.055772 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.055742    2550 generic.go:356] "Generic (PLEG): container finished" podID="fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08" containerID="27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa" exitCode=0
Sep 09 20:43:44.056233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.055785    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xsq4v"
Sep 09 20:43:44.056233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.055816    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xsq4v" event={"ID":"fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08","Type":"ContainerDied","Data":"27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa"}
Sep 09 20:43:44.056233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.055846    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xsq4v" event={"ID":"fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08","Type":"ContainerDied","Data":"57d69cf7508d9b83487cf9551522d1c019ae42a895d7f6e40a5bbcb60b87b27a"}
Sep 09 20:43:44.056233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.055873    2550 scope.go:117] "RemoveContainer" containerID="27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa"
Sep 09 20:43:44.064750 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.064734    2550 scope.go:117] "RemoveContainer" containerID="27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa"
Sep 09 20:43:44.064998 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:43:44.064978    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa\": container with ID starting with 27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa not found: ID does not exist" containerID="27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa"
Sep 09 20:43:44.065059 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.065009    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa"} err="failed to get container status \"27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa\": rpc error: code = NotFound desc = could not find container \"27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa\": container with ID starting with 27431035486de9da3b2e0b1e62a7c8a8e7ea11813cf9e2c2555e07401cc54efa not found: ID does not exist"
Sep 09 20:43:44.074593 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.074570    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xsq4v"]
Sep 09 20:43:44.076629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.076608    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xsq4v"]
Sep 09 20:43:44.278733 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.278702    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2c81d82-0265-490b-bc53-c21ef8eeed21" path="/var/lib/kubelet/pods/c2c81d82-0265-490b-bc53-c21ef8eeed21/volumes"
Sep 09 20:43:44.278906 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.278895    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df62eda8-9e07-4736-b595-029ef498b841" path="/var/lib/kubelet/pods/df62eda8-9e07-4736-b595-029ef498b841/volumes"
Sep 09 20:43:44.279079 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:44.279069    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08" path="/var/lib/kubelet/pods/fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08/volumes"
Sep 09 20:43:47.602058 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.602029    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:47.623289 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.623255    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7c78c8b878-cnjn9"]
Sep 09 20:43:47.623636 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.623623    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08" containerName="authorino"
Sep 09 20:43:47.623681 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.623639    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08" containerName="authorino"
Sep 09 20:43:47.623713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.623686    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa8d1a95-7ed3-4fe7-adc1-d3a8df569f08" containerName="authorino"
Sep 09 20:43:47.626403 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.626388    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.628919 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.628896    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2pbj7\""
Sep 09 20:43:47.629271 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.629219    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 20:43:47.629407 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.629377    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 20:43:47.629477 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.629426    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 20:43:47.635554 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.635530    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c78c8b878-cnjn9"]
Sep 09 20:43:47.669817 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.669782    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6s9l\" (UniqueName: \"kubernetes.io/projected/eade3681-8a13-4c80-9da8-6cd567d04266-kube-api-access-v6s9l\") pod \"maas-controller-7c78c8b878-cnjn9\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") " pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.669978 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.669845    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-webhook-cert\") pod \"maas-controller-7c78c8b878-cnjn9\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") " pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.669978 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.669897    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-metrics-certs\") pod \"maas-controller-7c78c8b878-cnjn9\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") " pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.698787 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.698758    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:47.758524 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.758488    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:47.770578 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.770535    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6s9l\" (UniqueName: \"kubernetes.io/projected/eade3681-8a13-4c80-9da8-6cd567d04266-kube-api-access-v6s9l\") pod \"maas-controller-7c78c8b878-cnjn9\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") " pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.770828 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.770803    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-webhook-cert\") pod \"maas-controller-7c78c8b878-cnjn9\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") " pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.770963 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.770866    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-metrics-certs\") pod \"maas-controller-7c78c8b878-cnjn9\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") " pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.774108 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.774053    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-webhook-cert\") pod \"maas-controller-7c78c8b878-cnjn9\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") " pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.774315 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.774296    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-metrics-certs\") pod \"maas-controller-7c78c8b878-cnjn9\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") " pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.780040 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.780016    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6s9l\" (UniqueName: \"kubernetes.io/projected/eade3681-8a13-4c80-9da8-6cd567d04266-kube-api-access-v6s9l\") pod \"maas-controller-7c78c8b878-cnjn9\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") " pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.783947 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.783926    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-845f9dff7c-h7v4k"]
Sep 09 20:43:47.787143 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.787126    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:47.796315 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.796289    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-h7v4k"]
Sep 09 20:43:47.872126 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.872031    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmpkb\" (UniqueName: \"kubernetes.io/projected/313705ea-49c9-478e-9b48-bf7ca727fb64-kube-api-access-jmpkb\") pod \"maas-controller-845f9dff7c-h7v4k\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") " pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:47.872126 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.872074    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-metrics-certs\") pod \"maas-controller-845f9dff7c-h7v4k\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") " pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:47.872347 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.872179    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-webhook-cert\") pod \"maas-controller-845f9dff7c-h7v4k\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") " pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:47.938022 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.937988    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:47.973518 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.973486    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-webhook-cert\") pod \"maas-controller-845f9dff7c-h7v4k\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") " pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:47.973669 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.973540    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmpkb\" (UniqueName: \"kubernetes.io/projected/313705ea-49c9-478e-9b48-bf7ca727fb64-kube-api-access-jmpkb\") pod \"maas-controller-845f9dff7c-h7v4k\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") " pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:47.973738 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.973715    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-metrics-certs\") pod \"maas-controller-845f9dff7c-h7v4k\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") " pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:47.976358 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.976335    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-metrics-certs\") pod \"maas-controller-845f9dff7c-h7v4k\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") " pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:47.976764 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.976744    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-webhook-cert\") pod \"maas-controller-845f9dff7c-h7v4k\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") " pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:47.980814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:47.980792    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmpkb\" (UniqueName: \"kubernetes.io/projected/313705ea-49c9-478e-9b48-bf7ca727fb64-kube-api-access-jmpkb\") pod \"maas-controller-845f9dff7c-h7v4k\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") " pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:48.098448 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:48.098412    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:48.211638 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:48.211613    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c78c8b878-cnjn9"]
Sep 09 20:43:48.213581 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:43:48.213552    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeade3681_8a13_4c80_9da8_6cd567d04266.slice/crio-fc228f9eef05facd723c4ad66a2980921a25de4dceb77868c29b279e2efdae80 WatchSource:0}: Error finding container fc228f9eef05facd723c4ad66a2980921a25de4dceb77868c29b279e2efdae80: Status 404 returned error can't find the container with id fc228f9eef05facd723c4ad66a2980921a25de4dceb77868c29b279e2efdae80
Sep 09 20:43:48.367718 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:48.367693    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-h7v4k"]
Sep 09 20:43:48.369412 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:43:48.369385    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod313705ea_49c9_478e_9b48_bf7ca727fb64.slice/crio-0eb53375ea646ad69f9eea21f385d19cd373fd654310d80a2994dcd86299c4a0 WatchSource:0}: Error finding container 0eb53375ea646ad69f9eea21f385d19cd373fd654310d80a2994dcd86299c4a0: Status 404 returned error can't find the container with id 0eb53375ea646ad69f9eea21f385d19cd373fd654310d80a2994dcd86299c4a0
Sep 09 20:43:49.088878 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:49.088837    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-845f9dff7c-h7v4k" event={"ID":"313705ea-49c9-478e-9b48-bf7ca727fb64","Type":"ContainerStarted","Data":"0eb53375ea646ad69f9eea21f385d19cd373fd654310d80a2994dcd86299c4a0"}
Sep 09 20:43:49.090846 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:49.090818    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c78c8b878-cnjn9" event={"ID":"eade3681-8a13-4c80-9da8-6cd567d04266","Type":"ContainerStarted","Data":"fc228f9eef05facd723c4ad66a2980921a25de4dceb77868c29b279e2efdae80"}
Sep 09 20:43:51.041715 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:51.041678    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:51.102449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:51.102414    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c78c8b878-cnjn9" event={"ID":"eade3681-8a13-4c80-9da8-6cd567d04266","Type":"ContainerStarted","Data":"fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e"}
Sep 09 20:43:51.102635 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:51.102572    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:43:51.103971 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:51.103949    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-845f9dff7c-h7v4k" event={"ID":"313705ea-49c9-478e-9b48-bf7ca727fb64","Type":"ContainerStarted","Data":"fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e"}
Sep 09 20:43:51.104070 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:51.103999    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:43:51.129935 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:51.129859    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7c78c8b878-cnjn9" podStartSLOduration=1.478965188 podStartE2EDuration="4.129844758s" totalImagesPullingTime="2.65087957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:43:51.129392856 +0000 UTC m=+691.280065201" watchObservedRunningTime="2026-09-09 20:43:51.129844758 +0000 UTC m=+691.280517101"
Sep 09 20:43:51.135993 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:51.135967    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:51.161153 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:43:51.161002    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-845f9dff7c-h7v4k" podStartSLOduration=1.658205014 podStartE2EDuration="4.160938117s" totalImagesPullingTime="2.502733103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:43:51.154033935 +0000 UTC m=+691.304706278" watchObservedRunningTime="2026-09-09 20:43:51.160938117 +0000 UTC m=+691.311610461"
Sep 09 20:44:02.110517 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.110487    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 20:44:02.110915 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.110556    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:44:02.182488 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.182450    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c78c8b878-cnjn9"]
Sep 09 20:44:02.182737 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.182692    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7c78c8b878-cnjn9" podUID="eade3681-8a13-4c80-9da8-6cd567d04266" containerName="manager" containerID="cri-o://fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e" gracePeriod=30
Sep 09 20:44:02.413414 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.413390    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:44:02.488345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.488319    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-webhook-cert\") pod \"eade3681-8a13-4c80-9da8-6cd567d04266\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") "
Sep 09 20:44:02.488499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.488356    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-metrics-certs\") pod \"eade3681-8a13-4c80-9da8-6cd567d04266\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") "
Sep 09 20:44:02.488499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.488449    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6s9l\" (UniqueName: \"kubernetes.io/projected/eade3681-8a13-4c80-9da8-6cd567d04266-kube-api-access-v6s9l\") pod \"eade3681-8a13-4c80-9da8-6cd567d04266\" (UID: \"eade3681-8a13-4c80-9da8-6cd567d04266\") "
Sep 09 20:44:02.490357 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.490319    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "eade3681-8a13-4c80-9da8-6cd567d04266" (UID: "eade3681-8a13-4c80-9da8-6cd567d04266"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:44:02.490357 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.490332    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eade3681-8a13-4c80-9da8-6cd567d04266" (UID: "eade3681-8a13-4c80-9da8-6cd567d04266"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:44:02.490535 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.490404    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eade3681-8a13-4c80-9da8-6cd567d04266-kube-api-access-v6s9l" (OuterVolumeSpecName: "kube-api-access-v6s9l") pod "eade3681-8a13-4c80-9da8-6cd567d04266" (UID: "eade3681-8a13-4c80-9da8-6cd567d04266"). InnerVolumeSpecName "kube-api-access-v6s9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:44:02.589964 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.589929    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6s9l\" (UniqueName: \"kubernetes.io/projected/eade3681-8a13-4c80-9da8-6cd567d04266-kube-api-access-v6s9l\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:44:02.589964 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.589955    2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-webhook-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:44:02.589964 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:02.589964    2550 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eade3681-8a13-4c80-9da8-6cd567d04266-metrics-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:44:03.165031 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:03.164997    2550 generic.go:356] "Generic (PLEG): container finished" podID="eade3681-8a13-4c80-9da8-6cd567d04266" containerID="fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e" exitCode=0
Sep 09 20:44:03.165486 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:03.165047    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c78c8b878-cnjn9"
Sep 09 20:44:03.165486 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:03.165094    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c78c8b878-cnjn9" event={"ID":"eade3681-8a13-4c80-9da8-6cd567d04266","Type":"ContainerDied","Data":"fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e"}
Sep 09 20:44:03.165486 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:03.165192    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c78c8b878-cnjn9" event={"ID":"eade3681-8a13-4c80-9da8-6cd567d04266","Type":"ContainerDied","Data":"fc228f9eef05facd723c4ad66a2980921a25de4dceb77868c29b279e2efdae80"}
Sep 09 20:44:03.165486 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:03.165228    2550 scope.go:117] "RemoveContainer" containerID="fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e"
Sep 09 20:44:03.174103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:03.174084    2550 scope.go:117] "RemoveContainer" containerID="fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e"
Sep 09 20:44:03.174384 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:44:03.174366    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e\": container with ID starting with fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e not found: ID does not exist" containerID="fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e"
Sep 09 20:44:03.174432 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:03.174393    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e"} err="failed to get container status \"fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e\": rpc error: code = NotFound desc = could not find container \"fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e\": container with ID starting with fbb31812d19b4be41dcc1e2f1621771065d29c290c7e15e54c30f9f93a6f7e5e not found: ID does not exist"
Sep 09 20:44:03.185278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:03.185254    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c78c8b878-cnjn9"]
Sep 09 20:44:03.189556 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:03.189531    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7c78c8b878-cnjn9"]
Sep 09 20:44:04.278567 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:04.278533    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eade3681-8a13-4c80-9da8-6cd567d04266" path="/var/lib/kubelet/pods/eade3681-8a13-4c80-9da8-6cd567d04266/volumes"
Sep 09 20:44:22.494379 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:22.494349    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:23.964352 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:23.964313    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:24.196735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.196694    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:24.298338 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.298235    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:24.363184 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.363140    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"]
Sep 09 20:44:24.363478 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.363466    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eade3681-8a13-4c80-9da8-6cd567d04266" containerName="manager"
Sep 09 20:44:24.363520 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.363480    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="eade3681-8a13-4c80-9da8-6cd567d04266" containerName="manager"
Sep 09 20:44:24.363559 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.363530    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="eade3681-8a13-4c80-9da8-6cd567d04266" containerName="manager"
Sep 09 20:44:24.368074 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.368050    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.371004 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.370984    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 20:44:24.371831 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.371805    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 20:44:24.371944 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.371841    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 20:44:24.372254 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.372238    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pzpjp\""
Sep 09 20:44:24.372360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.372285    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 20:44:24.383541 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.383518    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"]
Sep 09 20:44:24.392698 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.392679    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:24.469479 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.469447    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-metrics-certs\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.469643 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.469490    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf2c30b-351e-42ef-b1f1-602f8dee730b-openshift-service-ca\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.469643 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.469517    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgqtk\" (UniqueName: \"kubernetes.io/projected/caf2c30b-351e-42ef-b1f1-602f8dee730b-kube-api-access-pgqtk\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.469643 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.469551    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.570341 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.570261    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-metrics-certs\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.570341 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.570310    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf2c30b-351e-42ef-b1f1-602f8dee730b-openshift-service-ca\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.570341 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.570331    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgqtk\" (UniqueName: \"kubernetes.io/projected/caf2c30b-351e-42ef-b1f1-602f8dee730b-kube-api-access-pgqtk\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.570607 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.570358    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.570607 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:44:24.570462    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 09 20:44:24.570607 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:44:24.570530    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls podName:caf2c30b-351e-42ef-b1f1-602f8dee730b nodeName:}" failed. No retries permitted until 2026-09-09 20:44:25.070511027 +0000 UTC m=+725.221183350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls") pod "maas-api-6fc5cc7c7f-2kvd4" (UID: "caf2c30b-351e-42ef-b1f1-602f8dee730b") : secret "maas-api-serving-cert" not found
Sep 09 20:44:24.570870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.570850    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf2c30b-351e-42ef-b1f1-602f8dee730b-openshift-service-ca\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.572650 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.572629    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-metrics-certs\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.580363 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.580339    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgqtk\" (UniqueName: \"kubernetes.io/projected/caf2c30b-351e-42ef-b1f1-602f8dee730b-kube-api-access-pgqtk\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:24.718858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.718817    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:24.812412 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.812382    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:24.843061 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.842994    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"]
Sep 09 20:44:24.846101 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.846080    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"
Sep 09 20:44:24.848722 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.848699    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-l4zps\""
Sep 09 20:44:24.848816 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.848777    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 20:44:24.864820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.864796    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"]
Sep 09 20:44:24.872156 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.872127    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85871292-25fc-4b5d-ad75-c0df6e970f08-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8n4mh\" (UID: \"85871292-25fc-4b5d-ad75-c0df6e970f08\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"
Sep 09 20:44:24.872294 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.872274    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqdvw\" (UniqueName: \"kubernetes.io/projected/85871292-25fc-4b5d-ad75-c0df6e970f08-kube-api-access-pqdvw\") pod \"payload-processing-5b54b7fbb7-8n4mh\" (UID: \"85871292-25fc-4b5d-ad75-c0df6e970f08\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"
Sep 09 20:44:24.972790 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.972747    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqdvw\" (UniqueName: \"kubernetes.io/projected/85871292-25fc-4b5d-ad75-c0df6e970f08-kube-api-access-pqdvw\") pod \"payload-processing-5b54b7fbb7-8n4mh\" (UID: \"85871292-25fc-4b5d-ad75-c0df6e970f08\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"
Sep 09 20:44:24.973246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.972826    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85871292-25fc-4b5d-ad75-c0df6e970f08-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8n4mh\" (UID: \"85871292-25fc-4b5d-ad75-c0df6e970f08\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"
Sep 09 20:44:24.973707 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.973681    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85871292-25fc-4b5d-ad75-c0df6e970f08-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8n4mh\" (UID: \"85871292-25fc-4b5d-ad75-c0df6e970f08\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"
Sep 09 20:44:24.980812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:24.980787    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqdvw\" (UniqueName: \"kubernetes.io/projected/85871292-25fc-4b5d-ad75-c0df6e970f08-kube-api-access-pqdvw\") pod \"payload-processing-5b54b7fbb7-8n4mh\" (UID: \"85871292-25fc-4b5d-ad75-c0df6e970f08\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"
Sep 09 20:44:25.073679 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:25.073642    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:25.073841 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:44:25.073791    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 09 20:44:25.073885 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:44:25.073865    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls podName:caf2c30b-351e-42ef-b1f1-602f8dee730b nodeName:}" failed. No retries permitted until 2026-09-09 20:44:26.073846062 +0000 UTC m=+726.224518409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls") pod "maas-api-6fc5cc7c7f-2kvd4" (UID: "caf2c30b-351e-42ef-b1f1-602f8dee730b") : secret "maas-api-serving-cert" not found
Sep 09 20:44:25.156306 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:25.156226    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"
Sep 09 20:44:25.280644 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:25.280588    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:25.459447 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:25.459402    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"]
Sep 09 20:44:25.463391 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:44:25.463360    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85871292_25fc_4b5d_ad75_c0df6e970f08.slice/crio-c3b8e08399be0e1fd1152b15f969f9fa237177ccd130e80bff290b12a4d93b92 WatchSource:0}: Error finding container c3b8e08399be0e1fd1152b15f969f9fa237177ccd130e80bff290b12a4d93b92: Status 404 returned error can't find the container with id c3b8e08399be0e1fd1152b15f969f9fa237177ccd130e80bff290b12a4d93b92
Sep 09 20:44:26.082498 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:26.082453    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:26.085329 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:26.085302    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls\") pod \"maas-api-6fc5cc7c7f-2kvd4\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") " pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:26.178309 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:26.178272    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:26.293187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:26.293129    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh" event={"ID":"85871292-25fc-4b5d-ad75-c0df6e970f08","Type":"ContainerStarted","Data":"c3b8e08399be0e1fd1152b15f969f9fa237177ccd130e80bff290b12a4d93b92"}
Sep 09 20:44:26.511188 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:26.511147    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"]
Sep 09 20:44:26.514452 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:44:26.514424    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf2c30b_351e_42ef_b1f1_602f8dee730b.slice/crio-dffab14e8c2ca07a21f2e736920c690cbc336481aa551ff2d952d4baf0f98029 WatchSource:0}: Error finding container dffab14e8c2ca07a21f2e736920c690cbc336481aa551ff2d952d4baf0f98029: Status 404 returned error can't find the container with id dffab14e8c2ca07a21f2e736920c690cbc336481aa551ff2d952d4baf0f98029
Sep 09 20:44:26.516759 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:26.516497    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:44:27.301992 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:27.301954    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4" event={"ID":"caf2c30b-351e-42ef-b1f1-602f8dee730b","Type":"ContainerStarted","Data":"dffab14e8c2ca07a21f2e736920c690cbc336481aa551ff2d952d4baf0f98029"}
Sep 09 20:44:28.309404 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:28.309366    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh" event={"ID":"85871292-25fc-4b5d-ad75-c0df6e970f08","Type":"ContainerStarted","Data":"a17e502e3333a8b84bb6008fef3c651eae0123639e0d050f23f155df6b65c177"}
Sep 09 20:44:28.309792 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:28.309532    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"
Sep 09 20:44:28.329499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:28.329416    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh" podStartSLOduration=2.266927183 podStartE2EDuration="4.32939782s" totalImagesPullingTime="2.062470637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:44:28.328649262 +0000 UTC m=+728.479321618" watchObservedRunningTime="2026-09-09 20:44:28.32939782 +0000 UTC m=+728.480070165"
Sep 09 20:44:29.316140 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:29.316102    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4" event={"ID":"caf2c30b-351e-42ef-b1f1-602f8dee730b","Type":"ContainerStarted","Data":"1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818"}
Sep 09 20:44:29.316527 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:29.316274    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:29.335757 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:29.335698    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4" podStartSLOduration=3.170633665 podStartE2EDuration="5.33568171s" totalImagesPullingTime="2.165048045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:44:29.331907727 +0000 UTC m=+729.482580076" watchObservedRunningTime="2026-09-09 20:44:29.33568171 +0000 UTC m=+729.486354056"
Sep 09 20:44:29.750954 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:29.750875    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:34.343428 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:34.343393    2550 generic.go:356] "Generic (PLEG): container finished" podID="3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c" containerID="0ec38e1c5fcefeb840fac81706b7acba1bd3e1e459dc529b15ff8cbedfd7e4dd" exitCode=0
Sep 09 20:44:34.343835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:34.343468    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" event={"ID":"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c","Type":"ContainerDied","Data":"0ec38e1c5fcefeb840fac81706b7acba1bd3e1e459dc529b15ff8cbedfd7e4dd"}
Sep 09 20:44:34.343880 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:34.343864    2550 scope.go:117] "RemoveContainer" containerID="0ec38e1c5fcefeb840fac81706b7acba1bd3e1e459dc529b15ff8cbedfd7e4dd"
Sep 09 20:44:34.344829 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:34.344802    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:44:34.344886 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:34.344870    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:44:34.344922 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:34.344913    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:44:35.326385 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:35.326356    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 20:44:35.365535 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:35.365500    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" event={"ID":"3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c","Type":"ContainerStarted","Data":"04020cca909333369b3d5146d0e3adece75818fe5585586fb775a3329534a131"}
Sep 09 20:44:35.420369 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:35.420336    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:44:35.420684 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:35.420659    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-jgz67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:44:35.420779 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:35.420727    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" podUID="3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 09 20:44:36.381224 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:36.381187    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:36.420582 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:36.420542    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-jgz67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:44:36.420743 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:36.420611    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" podUID="3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 09 20:44:36.478861 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:36.478827    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:37.420372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:37.420329    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-jgz67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:44:37.420814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:37.420390    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" podUID="3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 09 20:44:38.419960 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:38.419926    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-jgz67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:44:38.420125 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:38.419993    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" podUID="3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 09 20:44:39.318844 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:39.318813    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-8n4mh"
Sep 09 20:44:39.420098 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:39.420065    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-jgz67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:44:39.420266 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:39.420126    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" podUID="3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 09 20:44:40.420490 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:40.420452    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-jgz67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:44:40.420935 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:40.420509    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67" podUID="3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 09 20:44:41.424589 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:41.424560    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:44:41.425047 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:41.424930    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:44:41.425521 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:41.425503    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jgz67"
Sep 09 20:44:46.726371 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:46.726336    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:53.627642 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:44:53.627599    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:12.643822 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:12.643782    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:20.690328 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.690294    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"]
Sep 09 20:45:20.693348 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.693333    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.696114 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.696094    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 20:45:20.696262 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.696242    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ddfsp\""
Sep 09 20:45:20.696363 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.696257    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 20:45:20.701836 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.696928    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 09 20:45:20.710915 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.710888    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"]
Sep 09 20:45:20.733574 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.733540    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:20.848553 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.848475    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.848553 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.848526    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.848835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.848649    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.848835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.848707    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fg9h\" (UniqueName: \"kubernetes.io/projected/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-kube-api-access-6fg9h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.848835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.848742    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.848835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.848811    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.949418 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.949337    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.949418 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.949377    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.949590 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.949514    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.949590 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.949568    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fg9h\" (UniqueName: \"kubernetes.io/projected/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-kube-api-access-6fg9h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.949671 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.949594    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.949671 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.949646    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.949843 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.949823    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.949908 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.949888    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.950030 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.950009    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.951675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.951657    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.952017 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.952001    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:20.957736 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:20.957715    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fg9h\" (UniqueName: \"kubernetes.io/projected/e12aa589-8f4c-4f5f-ac18-aef7f100ffec-kube-api-access-6fg9h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5f27f\" (UID: \"e12aa589-8f4c-4f5f-ac18-aef7f100ffec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:21.007289 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:21.007259    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:21.283817 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:21.283794    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"]
Sep 09 20:45:21.285122 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:45:21.285097    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode12aa589_8f4c_4f5f_ac18_aef7f100ffec.slice/crio-05e460d268259d67e2533702bc8e217840b1d4c7bc431cb73237c3bbbb2c3ff4 WatchSource:0}: Error finding container 05e460d268259d67e2533702bc8e217840b1d4c7bc431cb73237c3bbbb2c3ff4: Status 404 returned error can't find the container with id 05e460d268259d67e2533702bc8e217840b1d4c7bc431cb73237c3bbbb2c3ff4
Sep 09 20:45:21.603295 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:21.603212    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f" event={"ID":"e12aa589-8f4c-4f5f-ac18-aef7f100ffec","Type":"ContainerStarted","Data":"05e460d268259d67e2533702bc8e217840b1d4c7bc431cb73237c3bbbb2c3ff4"}
Sep 09 20:45:22.611326 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:22.611289    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f" event={"ID":"e12aa589-8f4c-4f5f-ac18-aef7f100ffec","Type":"ContainerStarted","Data":"e11cff34bbad4e28aecf88edbd6fe0152c07fcdf643a319adc68a028cd25e587"}
Sep 09 20:45:22.630967 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:22.630907    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f" podStartSLOduration=1.473845873 podStartE2EDuration="2.630885896s" totalImagesPullingTime="1.157040023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:45:22.628848255 +0000 UTC m=+782.779520599" watchObservedRunningTime="2026-09-09 20:45:22.630885896 +0000 UTC m=+782.781558237"
Sep 09 20:45:28.718603 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:28.718566    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:31.007824 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:31.007786    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:31.008394 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:31.008366    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:31.021263 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:31.021235    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:31.668978 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:31.668952    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5f27f"
Sep 09 20:45:41.820692 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:41.820656    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:49.744470 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:45:49.744385    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:21.454137 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:21.454100    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d6bdbd8d7-rjdjc"]
Sep 09 20:46:21.457344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:21.457321    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc"
Sep 09 20:46:21.464458 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:21.464039    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d6bdbd8d7-rjdjc"]
Sep 09 20:46:21.550417 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:21.550386    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz2sv\" (UniqueName: \"kubernetes.io/projected/8ff09826-3532-4a67-818a-7ff83588fc37-kube-api-access-hz2sv\") pod \"authorino-7d6bdbd8d7-rjdjc\" (UID: \"8ff09826-3532-4a67-818a-7ff83588fc37\") " pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc"
Sep 09 20:46:21.550572 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:21.550460    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ff09826-3532-4a67-818a-7ff83588fc37-tls-cert\") pod \"authorino-7d6bdbd8d7-rjdjc\" (UID: \"8ff09826-3532-4a67-818a-7ff83588fc37\") " pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc"
Sep 09 20:46:21.651740 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:21.651710    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ff09826-3532-4a67-818a-7ff83588fc37-tls-cert\") pod \"authorino-7d6bdbd8d7-rjdjc\" (UID: \"8ff09826-3532-4a67-818a-7ff83588fc37\") " pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc"
Sep 09 20:46:21.651877 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:21.651749    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz2sv\" (UniqueName: \"kubernetes.io/projected/8ff09826-3532-4a67-818a-7ff83588fc37-kube-api-access-hz2sv\") pod \"authorino-7d6bdbd8d7-rjdjc\" (UID: \"8ff09826-3532-4a67-818a-7ff83588fc37\") " pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc"
Sep 09 20:46:21.654021 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:21.654004    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ff09826-3532-4a67-818a-7ff83588fc37-tls-cert\") pod \"authorino-7d6bdbd8d7-rjdjc\" (UID: \"8ff09826-3532-4a67-818a-7ff83588fc37\") " pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc"
Sep 09 20:46:21.658730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:21.658703    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz2sv\" (UniqueName: \"kubernetes.io/projected/8ff09826-3532-4a67-818a-7ff83588fc37-kube-api-access-hz2sv\") pod \"authorino-7d6bdbd8d7-rjdjc\" (UID: \"8ff09826-3532-4a67-818a-7ff83588fc37\") " pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc"
Sep 09 20:46:21.767394 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:21.767370    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc"
Sep 09 20:46:22.036639 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:22.036611    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d6bdbd8d7-rjdjc"]
Sep 09 20:46:22.038841 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:46:22.038812    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ff09826_3532_4a67_818a_7ff83588fc37.slice/crio-a7620049128fb5571f69c1ad504f9b34f831365e747ff7f9f7532d6166731107 WatchSource:0}: Error finding container a7620049128fb5571f69c1ad504f9b34f831365e747ff7f9f7532d6166731107: Status 404 returned error can't find the container with id a7620049128fb5571f69c1ad504f9b34f831365e747ff7f9f7532d6166731107
Sep 09 20:46:22.925652 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:22.925618    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc" event={"ID":"8ff09826-3532-4a67-818a-7ff83588fc37","Type":"ContainerStarted","Data":"60245f2101aa0fa29e999605fb4492dd2b6f77516864f4e85deb40ca8b59a243"}
Sep 09 20:46:22.926047 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:22.925663    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc" event={"ID":"8ff09826-3532-4a67-818a-7ff83588fc37","Type":"ContainerStarted","Data":"a7620049128fb5571f69c1ad504f9b34f831365e747ff7f9f7532d6166731107"}
Sep 09 20:46:22.943519 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:22.943477    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d6bdbd8d7-rjdjc" podStartSLOduration=1.217777614 podStartE2EDuration="1.943462495s" totalImagesPullingTime="725.684881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:22.941489504 +0000 UTC m=+843.092161850" watchObservedRunningTime="2026-09-09 20:46:22.943462495 +0000 UTC m=+843.094134838"
Sep 09 20:46:22.969862 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:22.969831    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-759c79f5db-cqm8p"]
Sep 09 20:46:22.970058 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:22.970032    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-759c79f5db-cqm8p" podUID="baa32b51-d591-495e-b2a4-4eff0c55d850" containerName="authorino" containerID="cri-o://29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8" gracePeriod=30
Sep 09 20:46:23.227730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.227700    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-759c79f5db-cqm8p"
Sep 09 20:46:23.263157 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.263101    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/baa32b51-d591-495e-b2a4-4eff0c55d850-tls-cert\") pod \"baa32b51-d591-495e-b2a4-4eff0c55d850\" (UID: \"baa32b51-d591-495e-b2a4-4eff0c55d850\") "
Sep 09 20:46:23.263350 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.263219    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2xp2\" (UniqueName: \"kubernetes.io/projected/baa32b51-d591-495e-b2a4-4eff0c55d850-kube-api-access-d2xp2\") pod \"baa32b51-d591-495e-b2a4-4eff0c55d850\" (UID: \"baa32b51-d591-495e-b2a4-4eff0c55d850\") "
Sep 09 20:46:23.265791 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.265756    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baa32b51-d591-495e-b2a4-4eff0c55d850-kube-api-access-d2xp2" (OuterVolumeSpecName: "kube-api-access-d2xp2") pod "baa32b51-d591-495e-b2a4-4eff0c55d850" (UID: "baa32b51-d591-495e-b2a4-4eff0c55d850"). InnerVolumeSpecName "kube-api-access-d2xp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:46:23.274094 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.274059    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa32b51-d591-495e-b2a4-4eff0c55d850-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "baa32b51-d591-495e-b2a4-4eff0c55d850" (UID: "baa32b51-d591-495e-b2a4-4eff0c55d850"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:46:23.364190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.364092    2550 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/baa32b51-d591-495e-b2a4-4eff0c55d850-tls-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:46:23.364190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.364120    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2xp2\" (UniqueName: \"kubernetes.io/projected/baa32b51-d591-495e-b2a4-4eff0c55d850-kube-api-access-d2xp2\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:46:23.473601 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.473565    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:23.571708 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.571619    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:23.932443 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.932401    2550 generic.go:356] "Generic (PLEG): container finished" podID="baa32b51-d591-495e-b2a4-4eff0c55d850" containerID="29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8" exitCode=0
Sep 09 20:46:23.932443 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.932447    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-759c79f5db-cqm8p"
Sep 09 20:46:23.932936 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.932478    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-759c79f5db-cqm8p" event={"ID":"baa32b51-d591-495e-b2a4-4eff0c55d850","Type":"ContainerDied","Data":"29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8"}
Sep 09 20:46:23.932936 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.932518    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-759c79f5db-cqm8p" event={"ID":"baa32b51-d591-495e-b2a4-4eff0c55d850","Type":"ContainerDied","Data":"aa767e4d4b1b6d27897a5cf26e8aa8d0deb3497c25e8d762235db94c3be7abd2"}
Sep 09 20:46:23.932936 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.932542    2550 scope.go:117] "RemoveContainer" containerID="29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8"
Sep 09 20:46:23.943626 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.943593    2550 scope.go:117] "RemoveContainer" containerID="29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8"
Sep 09 20:46:23.943969 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:46:23.943920    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8\": container with ID starting with 29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8 not found: ID does not exist" containerID="29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8"
Sep 09 20:46:23.943969 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.943956    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8"} err="failed to get container status \"29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8\": rpc error: code = NotFound desc = could not find container \"29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8\": container with ID starting with 29d0326cd476ce09c1aa8623a8c934aeeb2e37d081a56da5678d379f2199ebd8 not found: ID does not exist"
Sep 09 20:46:23.955334 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.955308    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-759c79f5db-cqm8p"]
Sep 09 20:46:23.960835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:23.960805    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-759c79f5db-cqm8p"]
Sep 09 20:46:24.279365 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:24.279331    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baa32b51-d591-495e-b2a4-4eff0c55d850" path="/var/lib/kubelet/pods/baa32b51-d591-495e-b2a4-4eff0c55d850/volumes"
Sep 09 20:46:24.594500 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:24.594420    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:24.688654 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:24.688616    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:40.042316 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.042269    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:40.147727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.147690    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:40.247111 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.247078    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:40.411306 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.411237    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"]
Sep 09 20:46:40.411573 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.411560    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa32b51-d591-495e-b2a4-4eff0c55d850" containerName="authorino"
Sep 09 20:46:40.411631 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.411576    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa32b51-d591-495e-b2a4-4eff0c55d850" containerName="authorino"
Sep 09 20:46:40.411666 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.411639    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="baa32b51-d591-495e-b2a4-4eff0c55d850" containerName="authorino"
Sep 09 20:46:40.415956 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.415936    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.419468 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.419448    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-2b2343-openshift-default-dockercfg-g4qd7\""
Sep 09 20:46:40.446825 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.446797    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"]
Sep 09 20:46:40.469060 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.469033    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:40.492185 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.492057    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-credential-socket\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.492185 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.492104    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1f47c1-105f-4110-8d27-9aee8344090a-istiod-ca-cert\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.492185 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.492148    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-podinfo\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.492433 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.492237    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-token\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.492433 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.492268    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-socket\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.492433 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.492387    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbgl7\" (UniqueName: \"kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-kube-api-access-qbgl7\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.492433 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.492428    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-certs\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.492573 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.492466    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-data\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.492573 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.492527    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-envoy\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.590131 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.590102    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"]
Sep 09 20:46:40.593444 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.593421    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.593701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.593681    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbgl7\" (UniqueName: \"kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-kube-api-access-qbgl7\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.593764 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.593726    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-certs\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.593818 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.593767    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-data\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.593818 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.593808    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-envoy\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.593924 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.593831    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-credential-socket\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.593924 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.593859    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1f47c1-105f-4110-8d27-9aee8344090a-istiod-ca-cert\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.593924 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.593888    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-podinfo\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.593924 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.593917    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-token\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.594112 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.593946    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-socket\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.594112 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.594084    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-certs\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.594312 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.594293    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-data\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.594414 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.594393    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-socket\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.594664 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.594642    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-credential-socket\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.594822 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.594798    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1f47c1-105f-4110-8d27-9aee8344090a-istiod-ca-cert\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.596144 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.596122    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-envoy\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.596581 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.596556    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-podinfo\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.597552 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:46:40.597528    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.597644 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:46:40.597541    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-inf-a-c7643f-openshift-default-dockercfg-2fjv2\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c7643f-openshift-default-dockercfg-2fjv2\"" type="*v1.Secret"
Sep 09 20:46:40.604799 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.604714    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbgl7\" (UniqueName: \"kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-kube-api-access-qbgl7\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.605353 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.605314    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-token\") pod \"e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") " pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.631796 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.631755    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"]
Sep 09 20:46:40.679662 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.679585    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:40.680650 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.680627    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"]
Sep 09 20:46:40.685220 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.685201    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.687872 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.687851    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-933e9d-openshift-default-dockercfg-hztf4\""
Sep 09 20:46:40.694566 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.694546    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-socket\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.694671 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.694589    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-certs\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.694671 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.694607    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istiod-ca-cert\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.694671 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.694640    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8plfz\" (UniqueName: \"kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-kube-api-access-8plfz\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.694775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.694669    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-podinfo\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.694775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.694697    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-token\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.694775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.694717    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-data\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.694775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.694749    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-credential-socket\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.694896 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.694802    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-envoy\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.701338 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.701115    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"]
Sep 09 20:46:40.725866 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.725845    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:40.797065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.796765    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-data\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.797065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.796827    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-certs\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.797065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.796857    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istiod-ca-cert\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.797065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.796883    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8plfz\" (UniqueName: \"kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-kube-api-access-8plfz\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.796908    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-podinfo\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.799965    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-token\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.799997    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-data\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800022    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-socket\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800061    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-podinfo\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800091    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-certs\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800118    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-envoy\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800143    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qdsq\" (UniqueName: \"kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-kube-api-access-5qdsq\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800190    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-credential-socket\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800227    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istiod-ca-cert\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800258    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-envoy\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800283    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-token\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800346    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-socket\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.801501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800370    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-credential-socket\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.802318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.797478    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-certs\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.802318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.799900    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istiod-ca-cert\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.802318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.800962    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-data\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.802318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.801238    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-credential-socket\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.802318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.801647    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-socket\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.802318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.802184    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-podinfo\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.803590 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.803548    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-envoy\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.808236 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.808155    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8plfz\" (UniqueName: \"kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-kube-api-access-8plfz\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.814699 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.814552    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:40.815018 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.814999    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-token\") pod \"e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") " pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:40.901475 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901451    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-socket\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.901637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901482    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-podinfo\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.901637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901505    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-certs\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.901637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901521    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-envoy\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.901637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901536    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qdsq\" (UniqueName: \"kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-kube-api-access-5qdsq\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.901637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901557    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istiod-ca-cert\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.901637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901585    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-token\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.901637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901628    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-credential-socket\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.902033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901677    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-data\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.902033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901900    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-socket\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.902033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.901935    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-certs\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.902211 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.902085    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-credential-socket\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.902282 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.902218    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-data\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.902467 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.902443    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istiod-ca-cert\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.903952 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.903933    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-envoy\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.904205 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.904188    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-podinfo\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.911631 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.911602    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:40.912744 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.912723    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qdsq\" (UniqueName: \"kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-kube-api-access-5qdsq\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.912855 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.912836    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-token\") pod \"e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") " pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:40.995052 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:40.995024    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:41.008491 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.008468    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"]
Sep 09 20:46:41.010784 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:46:41.010762    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec1f47c1_105f_4110_8d27_9aee8344090a.slice/crio-91c2d8d4f0281f4dc6d7be230791d108be6ebec2b5c2c0299a87e48fcf786749 WatchSource:0}: Error finding container 91c2d8d4f0281f4dc6d7be230791d108be6ebec2b5c2c0299a87e48fcf786749: Status 404 returned error can't find the container with id 91c2d8d4f0281f4dc6d7be230791d108be6ebec2b5c2c0299a87e48fcf786749
Sep 09 20:46:41.012430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.012407    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:41.013832 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.013798    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:41.013919 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.013869    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:41.013919 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.013905    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:41.024397 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.024370    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" event={"ID":"ec1f47c1-105f-4110-8d27-9aee8344090a","Type":"ContainerStarted","Data":"91c2d8d4f0281f4dc6d7be230791d108be6ebec2b5c2c0299a87e48fcf786749"}
Sep 09 20:46:41.279971 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.279942    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"]
Sep 09 20:46:41.280809 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:46:41.280781    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f3bc29_b848_4d1b_8fe5_f06925a830de.slice/crio-aafe9d9f13cb180275625ac682e35b05ae359cbc68bc1fd96b52b88b6a90a7e0 WatchSource:0}: Error finding container aafe9d9f13cb180275625ac682e35b05ae359cbc68bc1fd96b52b88b6a90a7e0: Status 404 returned error can't find the container with id aafe9d9f13cb180275625ac682e35b05ae359cbc68bc1fd96b52b88b6a90a7e0
Sep 09 20:46:41.282922 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.282891    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:41.283001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.282953    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:41.283001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.282982    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:41.474213 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.474133    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:41.569701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.569623    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:41.924431 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.924356    2550 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Sep 09 20:46:41.924431 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:41.924424    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:42.021998 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.021963    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:42.030923 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.030888    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" event={"ID":"ec1f47c1-105f-4110-8d27-9aee8344090a","Type":"ContainerStarted","Data":"fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199"}
Sep 09 20:46:42.032635 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.032615    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" event={"ID":"e7f3bc29-b848-4d1b-8fe5-f06925a830de","Type":"ContainerStarted","Data":"0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a"}
Sep 09 20:46:42.032735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.032656    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" event={"ID":"e7f3bc29-b848-4d1b-8fe5-f06925a830de","Type":"ContainerStarted","Data":"aafe9d9f13cb180275625ac682e35b05ae359cbc68bc1fd96b52b88b6a90a7e0"}
Sep 09 20:46:42.098555 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.098495    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" podStartSLOduration=2.098476689 podStartE2EDuration="2.098476689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:42.073507973 +0000 UTC m=+862.224180316" watchObservedRunningTime="2026-09-09 20:46:42.098476689 +0000 UTC m=+862.249149033"
Sep 09 20:46:42.099485 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.099451    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" podStartSLOduration=2.099437533 podStartE2EDuration="2.099437533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:42.09745191 +0000 UTC m=+862.248124253" watchObservedRunningTime="2026-09-09 20:46:42.099437533 +0000 UTC m=+862.250109876"
Sep 09 20:46:42.126102 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.126078    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c7643f-openshift-default-dockercfg-2fjv2\""
Sep 09 20:46:42.204412 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.204388    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"]
Sep 09 20:46:42.206068 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:46:42.206042    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f61cfc8_70cc_4f5b_bb3e_19bcc5671c37.slice/crio-9f3afb6e942c8e3d5e61dd11ae9bf7f5daa3654a66ace8d4c0cc188d22971768 WatchSource:0}: Error finding container 9f3afb6e942c8e3d5e61dd11ae9bf7f5daa3654a66ace8d4c0cc188d22971768: Status 404 returned error can't find the container with id 9f3afb6e942c8e3d5e61dd11ae9bf7f5daa3654a66ace8d4c0cc188d22971768
Sep 09 20:46:42.208306 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.208276    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:42.208387 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.208336    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:42.208387 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.208366    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:42.726813 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.726774    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:42.731365 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.731342    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:42.995386 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:42.995305    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:43.000797 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:43.000766    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:43.047149 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:43.047117    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" event={"ID":"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37","Type":"ContainerStarted","Data":"ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0"}
Sep 09 20:46:43.047330 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:43.047199    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" event={"ID":"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37","Type":"ContainerStarted","Data":"9f3afb6e942c8e3d5e61dd11ae9bf7f5daa3654a66ace8d4c0cc188d22971768"}
Sep 09 20:46:43.047604 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:43.047576    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:43.047728 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:43.047619    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:43.048307 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:43.048288    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:46:43.048416 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:43.048333    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:46:43.069478 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:43.069436    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" podStartSLOduration=3.069423683 podStartE2EDuration="3.069423683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:43.067746943 +0000 UTC m=+863.218419286" watchObservedRunningTime="2026-09-09 20:46:43.069423683 +0000 UTC m=+863.220096026"
Sep 09 20:46:43.925453 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:43.925415    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:43.930249 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:43.930220    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:44.053361 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:44.053328    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:44.054097 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:44.054076    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:46:44.056106 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:44.056081    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:44.198025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:44.197933    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:44.303199 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:44.303150    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:45.017976 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.017946    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:45.150945 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.150915    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:45.227221 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.227188    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"]
Sep 09 20:46:45.234812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.234776    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.237199 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.237149    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-933e9d-serving-cert\""
Sep 09 20:46:45.245838 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.245785    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"]
Sep 09 20:46:45.282317 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.282201    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:45.346797 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.346755    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-maas-api-tls\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.347128 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.347090    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e608808-6c11-4bec-b102-0c8190042427-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.347330 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.347175    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtzwq\" (UniqueName: \"kubernetes.io/projected/4e608808-6c11-4bec-b102-0c8190042427-kube-api-access-vtzwq\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.347330 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.347218    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-metrics-certs\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.375541 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.375506    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:45.448117 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.448084    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-maas-api-tls\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.448117 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.448119    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e608808-6c11-4bec-b102-0c8190042427-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.448351 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.448141    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtzwq\" (UniqueName: \"kubernetes.io/projected/4e608808-6c11-4bec-b102-0c8190042427-kube-api-access-vtzwq\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.448351 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.448186    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-metrics-certs\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.448351 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:46:45.448266    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-serving-cert: secret "maas-api-e2e-worker-w4-933e9d-serving-cert" not found
Sep 09 20:46:45.448351 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:46:45.448339    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-maas-api-tls podName:4e608808-6c11-4bec-b102-0c8190042427 nodeName:}" failed. No retries permitted until 2026-09-09 20:46:45.948319076 +0000 UTC m=+866.098991397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-maas-api-tls") pod "maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw" (UID: "4e608808-6c11-4bec-b102-0c8190042427") : secret "maas-api-e2e-worker-w4-933e9d-serving-cert" not found
Sep 09 20:46:45.448932 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.448906    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e608808-6c11-4bec-b102-0c8190042427-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.450719 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.450689    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-metrics-certs\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.456893 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.456867    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtzwq\" (UniqueName: \"kubernetes.io/projected/4e608808-6c11-4bec-b102-0c8190042427-kube-api-access-vtzwq\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.759637 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.759598    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:45.920032 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.919992    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:45.953645 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.953608    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-maas-api-tls\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:45.957230 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:45.957204    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-maas-api-tls\") pod \"maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:46.021101 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.021013    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:46.149343 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.149309    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:46.247619 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.247561    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"]
Sep 09 20:46:46.254150 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.254114    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 20:46:46.256680 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.256482    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-933e9d\""
Sep 09 20:46:46.256680 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.256570    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-933e9d-dockercfg-4hzv9\""
Sep 09 20:46:46.256680 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.256689    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"]
Sep 09 20:46:46.356195 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.356085    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwv6m\" (UniqueName: \"kubernetes.io/projected/26957cab-ede5-4baf-ab09-1884410f4ed5-kube-api-access-gwv6m\") pod \"payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n\" (UID: \"26957cab-ede5-4baf-ab09-1884410f4ed5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 20:46:46.356195 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.356158    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26957cab-ede5-4baf-ab09-1884410f4ed5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n\" (UID: \"26957cab-ede5-4baf-ab09-1884410f4ed5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 20:46:46.464628 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.457160    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwv6m\" (UniqueName: \"kubernetes.io/projected/26957cab-ede5-4baf-ab09-1884410f4ed5-kube-api-access-gwv6m\") pod \"payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n\" (UID: \"26957cab-ede5-4baf-ab09-1884410f4ed5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 20:46:46.464628 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.457311    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26957cab-ede5-4baf-ab09-1884410f4ed5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n\" (UID: \"26957cab-ede5-4baf-ab09-1884410f4ed5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 20:46:46.464628 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.457419    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"]
Sep 09 20:46:46.464628 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.458603    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26957cab-ede5-4baf-ab09-1884410f4ed5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n\" (UID: \"26957cab-ede5-4baf-ab09-1884410f4ed5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 20:46:46.471271 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.471248    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwv6m\" (UniqueName: \"kubernetes.io/projected/26957cab-ede5-4baf-ab09-1884410f4ed5-kube-api-access-gwv6m\") pod \"payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n\" (UID: \"26957cab-ede5-4baf-ab09-1884410f4ed5\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 20:46:46.563967 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.563931    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:46.570008 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.569989    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 20:46:46.676890 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.676848    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:46.777798 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.777760    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:46.849349 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:46:46.849312    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26957cab_ede5_4baf_ab09_1884410f4ed5.slice/crio-278062589ee623a36e9e9d21d0585ed71bed8b9be2514aa6d9def1324ea386de WatchSource:0}: Error finding container 278062589ee623a36e9e9d21d0585ed71bed8b9be2514aa6d9def1324ea386de: Status 404 returned error can't find the container with id 278062589ee623a36e9e9d21d0585ed71bed8b9be2514aa6d9def1324ea386de
Sep 09 20:46:46.849700 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:46.849677    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"]
Sep 09 20:46:47.070686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:47.070652    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n" event={"ID":"26957cab-ede5-4baf-ab09-1884410f4ed5","Type":"ContainerStarted","Data":"2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68"}
Sep 09 20:46:47.071123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:47.070724    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n" event={"ID":"26957cab-ede5-4baf-ab09-1884410f4ed5","Type":"ContainerStarted","Data":"278062589ee623a36e9e9d21d0585ed71bed8b9be2514aa6d9def1324ea386de"}
Sep 09 20:46:47.071123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:47.070759    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 20:46:47.076729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:47.076701    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw" event={"ID":"4e608808-6c11-4bec-b102-0c8190042427","Type":"ContainerStarted","Data":"776e39ecee10ddd5e8ef237aa8cd41b79ca9044a1e147964f54aa4431311d7ca"}
Sep 09 20:46:47.132221 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:47.130785    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n" podStartSLOduration=1.130763891 podStartE2EDuration="1.130763891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:47.128463909 +0000 UTC m=+867.279136253" watchObservedRunningTime="2026-09-09 20:46:47.130763891 +0000 UTC m=+867.281436236"
Sep 09 20:46:47.519725 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:47.519687    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:47.619582 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:47.619547    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:47.997363 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:47.997319    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"]
Sep 09 20:46:48.002451 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.002426    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.004439 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.004409    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c7643f-serving-cert\""
Sep 09 20:46:48.014260 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.013453    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"]
Sep 09 20:46:48.039779 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.039741    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:48.074404 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.074374    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf6qm\" (UniqueName: \"kubernetes.io/projected/f921fe53-3f2a-4f42-82f2-6fc532a52117-kube-api-access-mf6qm\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.074810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.074419    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f921fe53-3f2a-4f42-82f2-6fc532a52117-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.074810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.074517    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.074810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.074580    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-metrics-certs\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.083676 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.083642    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw" event={"ID":"4e608808-6c11-4bec-b102-0c8190042427","Type":"ContainerStarted","Data":"88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9"}
Sep 09 20:46:48.083969 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.083954    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:48.102049 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.102007    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw" podStartSLOduration=2.487979012 podStartE2EDuration="3.101994263s" totalImagesPullingTime="614.015251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:48.099523763 +0000 UTC m=+868.250196107" watchObservedRunningTime="2026-09-09 20:46:48.101994263 +0000 UTC m=+868.252666606"
Sep 09 20:46:48.104087 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.104062    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:48.175231 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.175196    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf6qm\" (UniqueName: \"kubernetes.io/projected/f921fe53-3f2a-4f42-82f2-6fc532a52117-kube-api-access-mf6qm\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.175430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.175244    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f921fe53-3f2a-4f42-82f2-6fc532a52117-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.175430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.175341    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.175430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.175417    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-metrics-certs\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.175591 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:46:48.175565    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-serving-cert: secret "maas-api-e2e-inf-a-c7643f-serving-cert" not found
Sep 09 20:46:48.175659 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:46:48.175629    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls podName:f921fe53-3f2a-4f42-82f2-6fc532a52117 nodeName:}" failed. No retries permitted until 2026-09-09 20:46:48.675606758 +0000 UTC m=+868.826279083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls") pod "maas-api-e2e-inf-a-c7643f-587478dbb-w7m68" (UID: "f921fe53-3f2a-4f42-82f2-6fc532a52117") : secret "maas-api-e2e-inf-a-c7643f-serving-cert" not found
Sep 09 20:46:48.175970 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.175950    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f921fe53-3f2a-4f42-82f2-6fc532a52117-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.177785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.177768    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-metrics-certs\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.183441 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.183424    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf6qm\" (UniqueName: \"kubernetes.io/projected/f921fe53-3f2a-4f42-82f2-6fc532a52117-kube-api-access-mf6qm\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.584576 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.584536    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:48.684302 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.684265    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:48.689004 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.688977    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:48.689148 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:46:48.689133    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-serving-cert: secret "maas-api-e2e-inf-a-c7643f-serving-cert" not found
Sep 09 20:46:48.689229 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:46:48.689218    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls podName:f921fe53-3f2a-4f42-82f2-6fc532a52117 nodeName:}" failed. No retries permitted until 2026-09-09 20:46:49.689203641 +0000 UTC m=+869.839875962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls") pod "maas-api-e2e-inf-a-c7643f-587478dbb-w7m68" (UID: "f921fe53-3f2a-4f42-82f2-6fc532a52117") : secret "maas-api-e2e-inf-a-c7643f-serving-cert" not found
Sep 09 20:46:48.904278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:48.904147    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:49.123871 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:49.123832    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:49.509211 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:49.509152    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:49.698913 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:49.698876    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:49.701513 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:49.701480    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls\") pod \"maas-api-e2e-inf-a-c7643f-587478dbb-w7m68\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:49.787825 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:49.787735    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:49.820273 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:49.820238    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:49.898444 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:49.898389    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:49.986435 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:49.986401    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:50.085753 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.085665    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:50.108432 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.108406    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"]
Sep 09 20:46:50.110385 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:46:50.110362    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf921fe53_3f2a_4f42_82f2_6fc532a52117.slice/crio-16e1d567c106f14d05813c4211c2b3055cf4ac3cf9920e351f1c75f3a54266d8 WatchSource:0}: Error finding container 16e1d567c106f14d05813c4211c2b3055cf4ac3cf9920e351f1c75f3a54266d8: Status 404 returned error can't find the container with id 16e1d567c106f14d05813c4211c2b3055cf4ac3cf9920e351f1c75f3a54266d8
Sep 09 20:46:50.246272 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.246242    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"]
Sep 09 20:46:50.251720 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.251702    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.253950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.253930    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-04f122cb-openshift-default-dockercfg-xqmlt\""
Sep 09 20:46:50.264262 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.264241    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"]
Sep 09 20:46:50.282990 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.282966    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:50.305459 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.305430    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-podinfo\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.305556 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.305471    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkwpb\" (UniqueName: \"kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-kube-api-access-zkwpb\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.305556 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.305501    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-certs\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.305556 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.305547    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-socket\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.305730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.305634    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-data\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.305730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.305684    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-credential-socket\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.305830 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.305772    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2088dac5-9bab-4fbe-916b-5f1543a87333-istiod-ca-cert\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.305830 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.305810    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-token\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.305918 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.305881    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-envoy\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406434 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406353    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwpb\" (UniqueName: \"kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-kube-api-access-zkwpb\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406434 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406395    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-certs\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406434 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406423    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-socket\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406683 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406485    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-data\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406683 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406535    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-credential-socket\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406683 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406568    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2088dac5-9bab-4fbe-916b-5f1543a87333-istiod-ca-cert\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406683 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406605    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-token\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406893 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406678    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-envoy\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406893 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406727    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-podinfo\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406893 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406862    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-certs\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.406893 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406885    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-socket\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.407096 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.406929    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-credential-socket\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.407233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.407201    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-data\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.407374 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.407355    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2088dac5-9bab-4fbe-916b-5f1543a87333-istiod-ca-cert\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.409069 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.409041    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-envoy\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.409210 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.409158    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-podinfo\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.415434 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.415414    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-token\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.415676 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.415659    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkwpb\" (UniqueName: \"kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-kube-api-access-zkwpb\") pod \"e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") " pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.440658 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.440625    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:50.519476 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.519447    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:50.564690 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.564648    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:50.856322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.856295    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"]
Sep 09 20:46:50.857721 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:46:50.857691    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2088dac5_9bab_4fbe_916b_5f1543a87333.slice/crio-2a582581de23120edea9ee33c5c26ce89f5acfe4d8e30b0f11105f8d1936c32b WatchSource:0}: Error finding container 2a582581de23120edea9ee33c5c26ce89f5acfe4d8e30b0f11105f8d1936c32b: Status 404 returned error can't find the container with id 2a582581de23120edea9ee33c5c26ce89f5acfe4d8e30b0f11105f8d1936c32b
Sep 09 20:46:50.860319 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.860276    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:50.860444 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.860369    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:50.860444 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:50.860413    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:51.033661 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:51.033624    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:51.103990 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:51.103953    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68" event={"ID":"f921fe53-3f2a-4f42-82f2-6fc532a52117","Type":"ContainerStarted","Data":"7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4"}
Sep 09 20:46:51.104233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:51.104025    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:51.104233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:51.104049    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68" event={"ID":"f921fe53-3f2a-4f42-82f2-6fc532a52117","Type":"ContainerStarted","Data":"16e1d567c106f14d05813c4211c2b3055cf4ac3cf9920e351f1c75f3a54266d8"}
Sep 09 20:46:51.105839 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:51.105814    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" event={"ID":"2088dac5-9bab-4fbe-916b-5f1543a87333","Type":"ContainerStarted","Data":"72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45"}
Sep 09 20:46:51.105974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:51.105853    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" event={"ID":"2088dac5-9bab-4fbe-916b-5f1543a87333","Type":"ContainerStarted","Data":"2a582581de23120edea9ee33c5c26ce89f5acfe4d8e30b0f11105f8d1936c32b"}
Sep 09 20:46:51.125873 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:51.125825    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68" podStartSLOduration=3.589721552 podStartE2EDuration="4.12580663s" totalImagesPullingTime="536.085078ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:51.123342031 +0000 UTC m=+871.274014375" watchObservedRunningTime="2026-09-09 20:46:51.12580663 +0000 UTC m=+871.276478973"
Sep 09 20:46:51.142942 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:51.142888    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" podStartSLOduration=1.142873837 podStartE2EDuration="1.142873837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:51.141748743 +0000 UTC m=+871.292421087" watchObservedRunningTime="2026-09-09 20:46:51.142873837 +0000 UTC m=+871.293546182"
Sep 09 20:46:51.565475 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:51.565438    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:51.571678 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:51.571656    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:52.112198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:52.112150    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:52.113031 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:52.113009    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:46:52.519693 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:52.519656    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:52.614896 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:52.614862    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:54.101629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:54.101599    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 20:46:54.651190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:54.650793    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:54.744642 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:54.744610    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:54.889388 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:54.889290    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:55.228944 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.228875    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:55.324786 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.324752    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:55.519243 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.518562    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"]
Sep 09 20:46:55.540457 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.540428    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:46:55.543621 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.543591    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"]
Sep 09 20:46:55.562529 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.562497    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-04f122cb\""
Sep 09 20:46:55.563618 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.562782    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-04f122cb-dockercfg-82v6f\""
Sep 09 20:46:55.753737 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.753696    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd2c6\" (UniqueName: \"kubernetes.io/projected/c8c7ebbf-057a-43f3-be2f-a815efe35030-kube-api-access-bd2c6\") pod \"payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg\" (UID: \"c8c7ebbf-057a-43f3-be2f-a815efe35030\") " pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:46:55.753925 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.753773    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c7ebbf-057a-43f3-be2f-a815efe35030-plugins-config-volume\") pod \"payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg\" (UID: \"c8c7ebbf-057a-43f3-be2f-a815efe35030\") " pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:46:55.858226 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.857735    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd2c6\" (UniqueName: \"kubernetes.io/projected/c8c7ebbf-057a-43f3-be2f-a815efe35030-kube-api-access-bd2c6\") pod \"payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg\" (UID: \"c8c7ebbf-057a-43f3-be2f-a815efe35030\") " pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:46:55.858498 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.858477    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c7ebbf-057a-43f3-be2f-a815efe35030-plugins-config-volume\") pod \"payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg\" (UID: \"c8c7ebbf-057a-43f3-be2f-a815efe35030\") " pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:46:55.860652 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.859467    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c7ebbf-057a-43f3-be2f-a815efe35030-plugins-config-volume\") pod \"payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg\" (UID: \"c8c7ebbf-057a-43f3-be2f-a815efe35030\") " pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:46:55.888477 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.888419    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd2c6\" (UniqueName: \"kubernetes.io/projected/c8c7ebbf-057a-43f3-be2f-a815efe35030-kube-api-access-bd2c6\") pod \"payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg\" (UID: \"c8c7ebbf-057a-43f3-be2f-a815efe35030\") " pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:46:55.895854 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.895766    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:55.896484 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.896215    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:46:55.980778 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:55.980706    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:56.314541 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:56.309685    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"]
Sep 09 20:46:57.121914 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:57.121875    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:46:57.185209 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:57.184906    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg" event={"ID":"c8c7ebbf-057a-43f3-be2f-a815efe35030","Type":"ContainerStarted","Data":"589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85"}
Sep 09 20:46:57.185209 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:57.184968    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg" event={"ID":"c8c7ebbf-057a-43f3-be2f-a815efe35030","Type":"ContainerStarted","Data":"407a4c3515adcffb2944f5386c47ec4dfb94f3b8085e4e4a19dbb59ec313e7db"}
Sep 09 20:46:57.187939 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:57.187906    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:46:57.217182 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:57.215888    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg" podStartSLOduration=2.2158646969999998 podStartE2EDuration="2.215864697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:57.214034987 +0000 UTC m=+877.364707331" watchObservedRunningTime="2026-09-09 20:46:57.215864697 +0000 UTC m=+877.366537040"
Sep 09 20:46:57.698282 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:57.698247    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:57.794071 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:57.794039    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:58.088263 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:58.087338    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 20:46:58.597446 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:58.597179    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:58.693929 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:46:58.693892    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:00.094630 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:00.094344    2550 patch_prober.go:28] interesting pod/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 20:47:00.094630 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:00.094451    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 20:47:00.784670 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:00.784632    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:00.960703 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:00.960657    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:01.055954 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:01.055860    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:01.157251 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:01.157137    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:03.851126 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:03.790911    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:04.238231 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.238194    2550 generic.go:356] "Generic (PLEG): container finished" podID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerID="fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199" exitCode=0
Sep 09 20:47:04.238392 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.238233    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" event={"ID":"ec1f47c1-105f-4110-8d27-9aee8344090a","Type":"ContainerDied","Data":"fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199"}
Sep 09 20:47:04.238693 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.238671    2550 scope.go:117] "RemoveContainer" containerID="fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199"
Sep 09 20:47:04.239891 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.239857    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:04.239990 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.239937    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:04.239990 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.239973    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:04.240225 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.240207    2550 generic.go:356] "Generic (PLEG): container finished" podID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerID="0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a" exitCode=0
Sep 09 20:47:04.240309 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.240271    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" event={"ID":"e7f3bc29-b848-4d1b-8fe5-f06925a830de","Type":"ContainerDied","Data":"0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a"}
Sep 09 20:47:04.240622 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.240605    2550 scope.go:117] "RemoveContainer" containerID="0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a"
Sep 09 20:47:04.241766 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.241733    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:04.241872 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.241816    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:04.241928 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.241867    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:04.242235 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.242146    2550 generic.go:356] "Generic (PLEG): container finished" podID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerID="ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0" exitCode=0
Sep 09 20:47:04.242235 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.242199    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" event={"ID":"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37","Type":"ContainerDied","Data":"ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0"}
Sep 09 20:47:04.242606 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.242588    2550 scope.go:117] "RemoveContainer" containerID="ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0"
Sep 09 20:47:04.243822 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.243790    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:04.243904 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.243864    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:04.243952 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:04.243921    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:05.253271 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.253229    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" event={"ID":"e7f3bc29-b848-4d1b-8fe5-f06925a830de","Type":"ContainerStarted","Data":"199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d"}
Sep 09 20:47:05.257096 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.257058    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" event={"ID":"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37","Type":"ContainerStarted","Data":"61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678"}
Sep 09 20:47:05.261080 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.261048    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" event={"ID":"ec1f47c1-105f-4110-8d27-9aee8344090a","Type":"ContainerStarted","Data":"73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba"}
Sep 09 20:47:05.664796 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.664702    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:05.726336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.726304    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:47:05.726710 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.726687    2550 patch_prober.go:28] interesting pod/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 09 20:47:05.726861 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.726838    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 09 20:47:05.925567 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.925493    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:47:05.925828 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.925808    2550 patch_prober.go:28] interesting pod/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 09 20:47:05.925939 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.925922    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 09 20:47:05.995614 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.995579    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:47:05.995957 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.995933    2550 patch_prober.go:28] interesting pod/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:47:05.996120 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:05.996095    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 20:47:06.727400 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:06.727361    2550 patch_prober.go:28] interesting pod/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 09 20:47:06.728442 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:06.727954    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 09 20:47:06.783201 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:06.783154    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:06.925107 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:06.925028    2550 patch_prober.go:28] interesting pod/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 09 20:47:06.925442 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:06.925395    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 09 20:47:06.997234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:06.997119    2550 patch_prober.go:28] interesting pod/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:47:06.997234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:06.997203    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 20:47:07.727063 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:07.726628    2550 patch_prober.go:28] interesting pod/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 09 20:47:07.727063 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:07.726694    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 09 20:47:07.925675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:07.925639    2550 patch_prober.go:28] interesting pod/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 09 20:47:07.926290 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:07.926246    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 09 20:47:07.996083 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:07.995986    2550 patch_prober.go:28] interesting pod/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:47:07.996391 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:07.996357    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 20:47:08.197311 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:08.197273    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:47:08.658514 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:08.658469    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:08.726546 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:08.726489    2550 patch_prober.go:28] interesting pod/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 09 20:47:08.726814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:08.726787    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 09 20:47:08.921186 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:08.921070    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:08.925630 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:08.925602    2550 patch_prober.go:28] interesting pod/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 09 20:47:08.925877 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:08.925836    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 09 20:47:08.995547 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:08.995513    2550 patch_prober.go:28] interesting pod/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:47:08.995937 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:08.995794    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 20:47:09.357310 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:09.357276    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:09.461068 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:09.461035    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:09.557663 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:09.557623    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:09.727103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:09.727066    2550 patch_prober.go:28] interesting pod/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 09 20:47:09.727444 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:09.727409    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 09 20:47:09.925603 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:09.925565    2550 patch_prober.go:28] interesting pod/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 09 20:47:09.925843 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:09.925818    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 09 20:47:09.996459 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:09.996355    2550 patch_prober.go:28] interesting pod/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:47:09.997055 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:09.997022    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 20:47:10.288069 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:10.287984    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:10.726701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:10.726666    2550 patch_prober.go:28] interesting pod/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 09 20:47:10.727004 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:10.726976    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 09 20:47:10.928571 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:10.928537    2550 patch_prober.go:28] interesting pod/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 09 20:47:10.928903 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:10.928870    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 09 20:47:10.995987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:10.995889    2550 patch_prober.go:28] interesting pod/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:47:10.996277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:10.996226    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 20:47:11.510093 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:11.510044    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:11.726994 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:11.726959    2550 patch_prober.go:28] interesting pod/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 09 20:47:11.727336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:11.727287    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 09 20:47:11.925661 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:11.925584    2550 patch_prober.go:28] interesting pod/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 09 20:47:11.925661 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:11.925638    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 09 20:47:12.000648 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:12.000614    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:47:12.001547 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:12.001496    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:47:12.002499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:12.002466    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 20:47:12.572038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:12.571949    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:12.731614 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:12.730308    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:47:12.731614 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:12.731557    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:47:12.734597 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:12.734440    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 20:47:12.928843 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:12.928749    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:47:12.929160 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:12.929141    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:47:12.929981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:12.929961    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:47:13.483904 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:13.483866    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:14.120625 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:14.120586    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:14.577444 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:14.577391    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:14.683729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:14.683689    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:15.197711 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:15.197678    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:15.491390 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:15.491347    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:15.934047 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:15.933963    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:16.079462 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:16.079405    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:16.183537 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:16.183492    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:17.368810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:17.368775    2550 generic.go:356] "Generic (PLEG): container finished" podID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerID="72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45" exitCode=0
Sep 09 20:47:17.369239 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:17.368835    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" event={"ID":"2088dac5-9bab-4fbe-916b-5f1543a87333","Type":"ContainerDied","Data":"72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45"}
Sep 09 20:47:17.369239 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:17.369232    2550 scope.go:117] "RemoveContainer" containerID="72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45"
Sep 09 20:47:17.370293 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:17.370264    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:17.370378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:17.370329    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:17.370419 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:17.370378    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:18.376778 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:18.376743    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" event={"ID":"2088dac5-9bab-4fbe-916b-5f1543a87333","Type":"ContainerStarted","Data":"48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a"}
Sep 09 20:47:18.565632 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:18.565597    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:47:18.566620 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:18.566593    2550 patch_prober.go:28] interesting pod/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:47:18.566732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:18.566640    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 20:47:18.978523 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:18.978487    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:19.076520 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:19.076483    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:19.523999 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:19.523963    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:19.565940 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:19.565903    2550 patch_prober.go:28] interesting pod/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:47:19.566147 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:19.565969    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 20:47:20.304668 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:20.304638    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:47:20.304668 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:20.304658    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:47:20.322301 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:20.322280    2550 scope.go:117] "RemoveContainer" containerID="99f6d8b1d0e971717b5e946d3830b65eea0d65b20b7da4434e897c0b93f31fbf"
Sep 09 20:47:20.565092 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:20.565001    2550 patch_prober.go:28] interesting pod/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:47:20.565669 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:20.565629    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 20:47:20.674754 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:20.674722    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:20.750440 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:20.750407    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:20.852025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:20.851932    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:21.274775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:21.274735    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:21.378265 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:21.378230    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:21.565663 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:21.565574    2550 patch_prober.go:28] interesting pod/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:47:21.566285 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:21.566252    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 20:47:21.858252 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:21.858102    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:22.112488 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:22.112391    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:47:22.565280 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:22.565242    2550 patch_prober.go:28] interesting pod/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:47:22.565483 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:22.565316    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 20:47:23.114118 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:23.114083    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:23.565400 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:23.565368    2550 patch_prober.go:28] interesting pod/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:47:23.565572 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:23.565431    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 20:47:24.396788 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:24.396754    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:24.497181 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:24.497135    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:24.570023 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:24.569994    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:47:24.571027 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:24.571005    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:47:24.628803 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:24.628774    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:24.838078 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:24.838046    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:25.495948 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.495915    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:25.540705 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.540674    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"]
Sep 09 20:47:25.540924 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.540896    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg" podUID="c8c7ebbf-057a-43f3-be2f-a815efe35030" containerName="payload-processing" containerID="cri-o://589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85" gracePeriod=30
Sep 09 20:47:25.580559 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.580525    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:25.686465 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.686434    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:25.795574 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.795538    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:25.811064 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.811041    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:47:25.899886 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.899818    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:25.917727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.917698    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd2c6\" (UniqueName: \"kubernetes.io/projected/c8c7ebbf-057a-43f3-be2f-a815efe35030-kube-api-access-bd2c6\") pod \"c8c7ebbf-057a-43f3-be2f-a815efe35030\" (UID: \"c8c7ebbf-057a-43f3-be2f-a815efe35030\") "
Sep 09 20:47:25.917917 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.917900    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c7ebbf-057a-43f3-be2f-a815efe35030-plugins-config-volume\") pod \"c8c7ebbf-057a-43f3-be2f-a815efe35030\" (UID: \"c8c7ebbf-057a-43f3-be2f-a815efe35030\") "
Sep 09 20:47:25.918328 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.918302    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8c7ebbf-057a-43f3-be2f-a815efe35030-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c8c7ebbf-057a-43f3-be2f-a815efe35030" (UID: "c8c7ebbf-057a-43f3-be2f-a815efe35030"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:47:25.920074 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.920048    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8c7ebbf-057a-43f3-be2f-a815efe35030-kube-api-access-bd2c6" (OuterVolumeSpecName: "kube-api-access-bd2c6") pod "c8c7ebbf-057a-43f3-be2f-a815efe35030" (UID: "c8c7ebbf-057a-43f3-be2f-a815efe35030"). InnerVolumeSpecName "kube-api-access-bd2c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:47:25.948132 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.948102    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"]
Sep 09 20:47:25.990337 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:25.990300    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:26.019401 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.019360    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c7ebbf-057a-43f3-be2f-a815efe35030-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:26.019401 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.019395    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd2c6\" (UniqueName: \"kubernetes.io/projected/c8c7ebbf-057a-43f3-be2f-a815efe35030-kube-api-access-bd2c6\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:26.421340 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.421303    2550 generic.go:356] "Generic (PLEG): container finished" podID="c8c7ebbf-057a-43f3-be2f-a815efe35030" containerID="589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85" exitCode=0
Sep 09 20:47:26.421516 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.421348    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"
Sep 09 20:47:26.421516 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.421385    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg" event={"ID":"c8c7ebbf-057a-43f3-be2f-a815efe35030","Type":"ContainerDied","Data":"589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85"}
Sep 09 20:47:26.421516 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.421451    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg" event={"ID":"c8c7ebbf-057a-43f3-be2f-a815efe35030","Type":"ContainerDied","Data":"407a4c3515adcffb2944f5386c47ec4dfb94f3b8085e4e4a19dbb59ec313e7db"}
Sep 09 20:47:26.421516 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.421482    2550 scope.go:117] "RemoveContainer" containerID="589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85"
Sep 09 20:47:26.421732 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.421680    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy" containerID="cri-o://48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a" gracePeriod=30
Sep 09 20:47:26.430498 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.430474    2550 scope.go:117] "RemoveContainer" containerID="589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85"
Sep 09 20:47:26.430792 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:47:26.430773    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85\": container with ID starting with 589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85 not found: ID does not exist" containerID="589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85"
Sep 09 20:47:26.430877 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.430801    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85"} err="failed to get container status \"589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85\": rpc error: code = NotFound desc = could not find container \"589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85\": container with ID starting with 589dad6da8b0f39c7f6f4a38576f9516d3b6adaa1cd903f61cfb7448392beb85 not found: ID does not exist"
Sep 09 20:47:26.454664 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.454634    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:26.487826 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.487794    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"]
Sep 09 20:47:26.497643 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.497620    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-04f122cb-6fd47bc995-cxgcg"]
Sep 09 20:47:26.560679 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.560624    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:26.713619 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:26.713590    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:27.036035 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.036003    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"]
Sep 09 20:47:27.036429 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.036416    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8c7ebbf-057a-43f3-be2f-a815efe35030" containerName="payload-processing"
Sep 09 20:47:27.036488 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.036431    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8c7ebbf-057a-43f3-be2f-a815efe35030" containerName="payload-processing"
Sep 09 20:47:27.036521 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.036510    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8c7ebbf-057a-43f3-be2f-a815efe35030" containerName="payload-processing"
Sep 09 20:47:27.039189 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.039157    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.044457 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.044437    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-5333ad9e-gw-openshift-default-dockercfg-4vbw4\""
Sep 09 20:47:27.072155 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.072133    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"]
Sep 09 20:47:27.130729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.130238    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-token\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.130729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.130288    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-envoy\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.130729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.130325    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-podinfo\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.130729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.130364    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-credential-socket\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.130729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.130430    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-data\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.130729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.130463    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwrxb\" (UniqueName: \"kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-kube-api-access-dwrxb\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.130729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.130543    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-socket\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.130729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.130588    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-certs\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.130729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.130619    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd4555ea-212d-4cb2-9cd6-938f120fda04-istiod-ca-cert\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.200372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.200335    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:27.231967 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.231937    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-certs\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.232123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.231983    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd4555ea-212d-4cb2-9cd6-938f120fda04-istiod-ca-cert\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.232123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.232019    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-token\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.232123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.232037    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-envoy\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.232123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.232064    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-podinfo\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.232123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.232118    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-credential-socket\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.232436 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.232411    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-certs\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.232504 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.232457    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-data\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.232504 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.232499    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwrxb\" (UniqueName: \"kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-kube-api-access-dwrxb\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.232640 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.232603    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-socket\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.233018 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.232778    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd4555ea-212d-4cb2-9cd6-938f120fda04-istiod-ca-cert\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.233018 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.232974    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-data\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.233018 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.233009    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-credential-socket\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.233181 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.233026    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-socket\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.234560 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.234543    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-envoy\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.234977 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.234960    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-podinfo\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.248121 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.248100    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-token\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.248694 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.248672    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwrxb\" (UniqueName: \"kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-kube-api-access-dwrxb\") pod \"e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") " pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.349190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.349082    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:27.380047 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.380014    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:27.645961 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.645936    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"]
Sep 09 20:47:27.648245 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:47:27.648220    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4555ea_212d_4cb2_9cd6_938f120fda04.slice/crio-32225cd5bc1874de32ab6c1448b5de3a195e0af376660de0b958e6a6ced77abe WatchSource:0}: Error finding container 32225cd5bc1874de32ab6c1448b5de3a195e0af376660de0b958e6a6ced77abe: Status 404 returned error can't find the container with id 32225cd5bc1874de32ab6c1448b5de3a195e0af376660de0b958e6a6ced77abe
Sep 09 20:47:27.650716 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.650683    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:27.650802 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.650746    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:27.650802 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:27.650773    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:28.279354 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:28.279322    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8c7ebbf-057a-43f3-be2f-a815efe35030" path="/var/lib/kubelet/pods/c8c7ebbf-057a-43f3-be2f-a815efe35030/volumes"
Sep 09 20:47:28.438071 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:28.438036    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" event={"ID":"fd4555ea-212d-4cb2-9cd6-938f120fda04","Type":"ContainerStarted","Data":"c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e"}
Sep 09 20:47:28.438246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:28.438091    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" event={"ID":"fd4555ea-212d-4cb2-9cd6-938f120fda04","Type":"ContainerStarted","Data":"32225cd5bc1874de32ab6c1448b5de3a195e0af376660de0b958e6a6ced77abe"}
Sep 09 20:47:28.482812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:28.482757    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" podStartSLOduration=2.482739846 podStartE2EDuration="2.482739846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:47:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:47:28.479424404 +0000 UTC m=+908.630096747" watchObservedRunningTime="2026-09-09 20:47:28.482739846 +0000 UTC m=+908.633412266"
Sep 09 20:47:29.349945 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:29.349910    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:29.355295 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:29.355273    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:29.443658 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:29.443626    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:29.444639 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:29.444620    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:29.960933 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:29.960894    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:30.077839 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:30.077795    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:30.298620 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:30.298522    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:30.402704 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:30.402663    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:30.659613 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:30.659530    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:30.981493 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:30.981459    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:31.230445 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.230406    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:31.235178 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.235079    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"]
Sep 09 20:47:31.237861 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.237840    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:31.263714 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:47:31.263681    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="e56604e7-403a-4d12-a8ae-930c7c9882f2" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:31.263714 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:47:31.263689    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-e2e-migrate-5333ad9e-dockercfg-zkqkq\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-5333ad9e-dockercfg-zkqkq\"" type="*v1.Secret"
Sep 09 20:47:31.264993 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.264971    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-5333ad9e\""
Sep 09 20:47:31.364926 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.364883    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"]
Sep 09 20:47:31.372301 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.372269    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l9kr\" (UniqueName: \"kubernetes.io/projected/e56604e7-403a-4d12-a8ae-930c7c9882f2-kube-api-access-7l9kr\") pod \"payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv\" (UID: \"e56604e7-403a-4d12-a8ae-930c7c9882f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:31.372424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.372316    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e56604e7-403a-4d12-a8ae-930c7c9882f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv\" (UID: \"e56604e7-403a-4d12-a8ae-930c7c9882f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:31.474071 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.474023    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9kr\" (UniqueName: \"kubernetes.io/projected/e56604e7-403a-4d12-a8ae-930c7c9882f2-kube-api-access-7l9kr\") pod \"payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv\" (UID: \"e56604e7-403a-4d12-a8ae-930c7c9882f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:31.474455 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.474082    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e56604e7-403a-4d12-a8ae-930c7c9882f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv\" (UID: \"e56604e7-403a-4d12-a8ae-930c7c9882f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:31.475290 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.475266    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e56604e7-403a-4d12-a8ae-930c7c9882f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv\" (UID: \"e56604e7-403a-4d12-a8ae-930c7c9882f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:31.500675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.500603    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l9kr\" (UniqueName: \"kubernetes.io/projected/e56604e7-403a-4d12-a8ae-930c7c9882f2-kube-api-access-7l9kr\") pod \"payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv\" (UID: \"e56604e7-403a-4d12-a8ae-930c7c9882f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:31.680740 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.680708    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:47:31.777440 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777341    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-podinfo\") pod \"2088dac5-9bab-4fbe-916b-5f1543a87333\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") "
Sep 09 20:47:31.777440 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777394    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-data\") pod \"2088dac5-9bab-4fbe-916b-5f1543a87333\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") "
Sep 09 20:47:31.777440 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777424    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-envoy\") pod \"2088dac5-9bab-4fbe-916b-5f1543a87333\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") "
Sep 09 20:47:31.777744 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777469    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-credential-socket\") pod \"2088dac5-9bab-4fbe-916b-5f1543a87333\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") "
Sep 09 20:47:31.777744 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777511    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2088dac5-9bab-4fbe-916b-5f1543a87333-istiod-ca-cert\") pod \"2088dac5-9bab-4fbe-916b-5f1543a87333\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") "
Sep 09 20:47:31.777744 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777597    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkwpb\" (UniqueName: \"kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-kube-api-access-zkwpb\") pod \"2088dac5-9bab-4fbe-916b-5f1543a87333\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") "
Sep 09 20:47:31.777744 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777623    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-socket\") pod \"2088dac5-9bab-4fbe-916b-5f1543a87333\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") "
Sep 09 20:47:31.777744 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777673    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-token\") pod \"2088dac5-9bab-4fbe-916b-5f1543a87333\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") "
Sep 09 20:47:31.777744 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777712    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-certs\") pod \"2088dac5-9bab-4fbe-916b-5f1543a87333\" (UID: \"2088dac5-9bab-4fbe-916b-5f1543a87333\") "
Sep 09 20:47:31.778045 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777794    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-data" (OuterVolumeSpecName: "istio-data") pod "2088dac5-9bab-4fbe-916b-5f1543a87333" (UID: "2088dac5-9bab-4fbe-916b-5f1543a87333"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:47:31.778045 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.777821    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2088dac5-9bab-4fbe-916b-5f1543a87333" (UID: "2088dac5-9bab-4fbe-916b-5f1543a87333"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:47:31.778160 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.778103    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:31.778160 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.778109    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2088dac5-9bab-4fbe-916b-5f1543a87333" (UID: "2088dac5-9bab-4fbe-916b-5f1543a87333"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:47:31.778160 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.778123    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:31.778160 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.778151    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2088dac5-9bab-4fbe-916b-5f1543a87333-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2088dac5-9bab-4fbe-916b-5f1543a87333" (UID: "2088dac5-9bab-4fbe-916b-5f1543a87333"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:47:31.778414 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.778368    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2088dac5-9bab-4fbe-916b-5f1543a87333" (UID: "2088dac5-9bab-4fbe-916b-5f1543a87333"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:47:31.780496 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.780465    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-kube-api-access-zkwpb" (OuterVolumeSpecName: "kube-api-access-zkwpb") pod "2088dac5-9bab-4fbe-916b-5f1543a87333" (UID: "2088dac5-9bab-4fbe-916b-5f1543a87333"). InnerVolumeSpecName "kube-api-access-zkwpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:47:31.780496 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.780485    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2088dac5-9bab-4fbe-916b-5f1543a87333" (UID: "2088dac5-9bab-4fbe-916b-5f1543a87333"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:47:31.780673 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.780510    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-token" (OuterVolumeSpecName: "istio-token") pod "2088dac5-9bab-4fbe-916b-5f1543a87333" (UID: "2088dac5-9bab-4fbe-916b-5f1543a87333"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:47:31.780673 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.780581    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2088dac5-9bab-4fbe-916b-5f1543a87333" (UID: "2088dac5-9bab-4fbe-916b-5f1543a87333"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:47:31.878775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.878741    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:31.878775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.878771    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:31.878775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.878780    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:31.879001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.878788    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:31.879001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.878796    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2088dac5-9bab-4fbe-916b-5f1543a87333-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:31.879001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.878803    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2088dac5-9bab-4fbe-916b-5f1543a87333-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:31.879001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:31.878812    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkwpb\" (UniqueName: \"kubernetes.io/projected/2088dac5-9bab-4fbe-916b-5f1543a87333-kube-api-access-zkwpb\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:47:32.468037 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.467993    2550 generic.go:356] "Generic (PLEG): container finished" podID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerID="48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a" exitCode=0
Sep 09 20:47:32.468234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.468045    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"
Sep 09 20:47:32.468234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.468067    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" event={"ID":"2088dac5-9bab-4fbe-916b-5f1543a87333","Type":"ContainerDied","Data":"48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a"}
Sep 09 20:47:32.468234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.468131    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz" event={"ID":"2088dac5-9bab-4fbe-916b-5f1543a87333","Type":"ContainerDied","Data":"2a582581de23120edea9ee33c5c26ce89f5acfe4d8e30b0f11105f8d1936c32b"}
Sep 09 20:47:32.468234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.468186    2550 scope.go:117] "RemoveContainer" containerID="48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a"
Sep 09 20:47:32.490795 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.488482    2550 scope.go:117] "RemoveContainer" containerID="72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45"
Sep 09 20:47:32.503861 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.503841    2550 scope.go:117] "RemoveContainer" containerID="48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a"
Sep 09 20:47:32.504418 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:47:32.504396    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a\": container with ID starting with 48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a not found: ID does not exist" containerID="48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a"
Sep 09 20:47:32.504418 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.504427    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a"} err="failed to get container status \"48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a\": rpc error: code = NotFound desc = could not find container \"48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a\": container with ID starting with 48ea59952cf8b700aaf8ebbd9627ffc91f24ce27c87fa13b8601e0c4bcfe720a not found: ID does not exist"
Sep 09 20:47:32.504418 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.504447    2550 scope.go:117] "RemoveContainer" containerID="72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45"
Sep 09 20:47:32.504746 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:47:32.504729    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45\": container with ID starting with 72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45 not found: ID does not exist" containerID="72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45"
Sep 09 20:47:32.504790 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.504759    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45"} err="failed to get container status \"72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45\": rpc error: code = NotFound desc = could not find container \"72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45\": container with ID starting with 72cbb181122f64bb92c47751921877cefdac0222bdf641e367920d7d23183a45 not found: ID does not exist"
Sep 09 20:47:32.519380 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.519357    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"]
Sep 09 20:47:32.521716 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.521698    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-04f122cb-openshift-default-564ffdcbb8-d6czz"]
Sep 09 20:47:32.550892 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.550819    2550 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Sep 09 20:47:32.550892 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.550881    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:32.820740 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.820671    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-5333ad9e-dockercfg-zkqkq\""
Sep 09 20:47:32.875853 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.875817    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"]
Sep 09 20:47:32.881541 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:47:32.881502    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode56604e7_403a_4d12_a8ae_930c7c9882f2.slice/crio-1eb6ea113f0124dbda42e7df1b74503009735ae3949b3ab6cbc9044243e23084 WatchSource:0}: Error finding container 1eb6ea113f0124dbda42e7df1b74503009735ae3949b3ab6cbc9044243e23084: Status 404 returned error can't find the container with id 1eb6ea113f0124dbda42e7df1b74503009735ae3949b3ab6cbc9044243e23084
Sep 09 20:47:32.951048 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.951023    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"]
Sep 09 20:47:32.951730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.951713    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy"
Sep 09 20:47:32.951785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.951733    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy"
Sep 09 20:47:32.951785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.951766    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy"
Sep 09 20:47:32.951785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.951773    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy"
Sep 09 20:47:32.951898 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.951883    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy"
Sep 09 20:47:32.955578 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.955547    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:32.962940 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.962922    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-933e9d\"/\"default-dockercfg-85jbr\""
Sep 09 20:47:32.964018 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.963998    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-933e9d\"/\"simulator-w4-933e9d-kserve-self-signed-certs\""
Sep 09 20:47:32.972815 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.972789    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-933e9d\"/\"openshift-service-ca.crt\""
Sep 09 20:47:32.973010 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.972987    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-933e9d\"/\"kube-root-ca.crt\""
Sep 09 20:47:32.983484 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:32.983461    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"]
Sep 09 20:47:33.090692 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.090603    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwzs5\" (UniqueName: \"kubernetes.io/projected/2b733fc0-c554-4780-aa61-353b66ac36eb-kube-api-access-kwzs5\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.090692 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.090654    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-home\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.090692 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.090674    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b733fc0-c554-4780-aa61-353b66ac36eb-tls-certs\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.090933 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.090725    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-dshm\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.090933 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.090744    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-model-cache\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.090933 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.090839    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-tmp-dir\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.191674 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.191636    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-dshm\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.191874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.191691    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-model-cache\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.191874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.191778    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-tmp-dir\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.191874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.191832    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwzs5\" (UniqueName: \"kubernetes.io/projected/2b733fc0-c554-4780-aa61-353b66ac36eb-kube-api-access-kwzs5\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.192065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.191880    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-home\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.192126 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.192076    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b733fc0-c554-4780-aa61-353b66ac36eb-tls-certs\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.192126 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.192114    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-model-cache\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.192259 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.192192    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-home\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.192259 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.192222    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-tmp-dir\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.194014 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.193991    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-dshm\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.194361 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.194343    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b733fc0-c554-4780-aa61-353b66ac36eb-tls-certs\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.214985 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.214948    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwzs5\" (UniqueName: \"kubernetes.io/projected/2b733fc0-c554-4780-aa61-353b66ac36eb-kube-api-access-kwzs5\") pod \"simulator-w4-933e9d-kserve-67f88dd894-lvd4j\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") " pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.266617 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.266578    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:33.273956 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.273919    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:33.475920 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.475888    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv" event={"ID":"e56604e7-403a-4d12-a8ae-930c7c9882f2","Type":"ContainerStarted","Data":"d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d"}
Sep 09 20:47:33.476105 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.475939    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv" event={"ID":"e56604e7-403a-4d12-a8ae-930c7c9882f2","Type":"ContainerStarted","Data":"1eb6ea113f0124dbda42e7df1b74503009735ae3949b3ab6cbc9044243e23084"}
Sep 09 20:47:33.476105 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.476005    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:33.504555 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.504502    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv" podStartSLOduration=2.504487552 podStartE2EDuration="2.504487552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:47:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:47:33.500553169 +0000 UTC m=+913.651225512" watchObservedRunningTime="2026-09-09 20:47:33.504487552 +0000 UTC m=+913.655159895"
Sep 09 20:47:33.561670 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:33.561486    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"]
Sep 09 20:47:33.564240 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:47:33.564214    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b733fc0_c554_4780_aa61_353b66ac36eb.slice/crio-0d0b906a6653156665466241486e22476e331dab2f1fb4333d23ea556462f2ee WatchSource:0}: Error finding container 0d0b906a6653156665466241486e22476e331dab2f1fb4333d23ea556462f2ee: Status 404 returned error can't find the container with id 0d0b906a6653156665466241486e22476e331dab2f1fb4333d23ea556462f2ee
Sep 09 20:47:34.120039 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:34.120007    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:34.271371 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:34.271334    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:34.280182 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:34.280138    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" path="/var/lib/kubelet/pods/2088dac5-9bab-4fbe-916b-5f1543a87333/volumes"
Sep 09 20:47:34.377809 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:34.377727    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:34.484704 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:34.484664    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j" event={"ID":"2b733fc0-c554-4780-aa61-353b66ac36eb","Type":"ContainerStarted","Data":"56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a"}
Sep 09 20:47:34.484871 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:34.484711    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j" event={"ID":"2b733fc0-c554-4780-aa61-353b66ac36eb","Type":"ContainerStarted","Data":"0d0b906a6653156665466241486e22476e331dab2f1fb4333d23ea556462f2ee"}
Sep 09 20:47:34.507782 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:34.507751    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:34.508156 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:34.508120    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j" podStartSLOduration=2.508105594 podStartE2EDuration="2.508105594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:47:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:47:34.504812808 +0000 UTC m=+914.655485152" watchObservedRunningTime="2026-09-09 20:47:34.508105594 +0000 UTC m=+914.658777937"
Sep 09 20:47:41.752259 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:41.752223    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:41.998854 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:41.998818    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:42.228938 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:42.228890    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:43.267000 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:43.266970    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:43.267407 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:43.267318    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:43.279631 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:43.279606    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:43.573623 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:43.573536    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 20:47:43.892402 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:43.892310    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:44.486999 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:44.486973    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:47:50.593137 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:50.593062    2550 generic.go:356] "Generic (PLEG): container finished" podID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerID="c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e" exitCode=0
Sep 09 20:47:50.593557 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:50.593133    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" event={"ID":"fd4555ea-212d-4cb2-9cd6-938f120fda04","Type":"ContainerDied","Data":"c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e"}
Sep 09 20:47:50.593557 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:50.593542    2550 scope.go:117] "RemoveContainer" containerID="c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e"
Sep 09 20:47:50.595264 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:50.595226    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:50.595556 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:50.595532    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:50.595747 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:50.595727    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:51.600569 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:51.600533    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" event={"ID":"fd4555ea-212d-4cb2-9cd6-938f120fda04","Type":"ContainerStarted","Data":"ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9"}
Sep 09 20:47:52.136020 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:52.135983    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:52.349605 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:52.349511    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:52.349795 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:52.349641    2550 patch_prober.go:28] interesting pod/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 09 20:47:52.349795 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:52.349701    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 09 20:47:53.350039 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:53.349551    2550 patch_prober.go:28] interesting pod/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 09 20:47:53.350039 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:53.349612    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 09 20:47:54.350379 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:54.350341    2550 patch_prober.go:28] interesting pod/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 09 20:47:54.351314 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:54.350922    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 09 20:47:55.349469 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:55.349433    2550 patch_prober.go:28] interesting pod/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 09 20:47:55.349763 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:55.349730    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 09 20:47:56.349696 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:56.349646    2550 patch_prober.go:28] interesting pod/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 09 20:47:56.350301 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:56.350262    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 09 20:47:57.354221 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:57.354182    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:57.354652 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:57.354466    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:47:57.355113 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:47:57.355096    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:48:01.691689 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:01.691651    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:01.712635 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:01.712604    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"]
Sep 09 20:48:01.712879 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:01.712835    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv" podUID="e56604e7-403a-4d12-a8ae-930c7c9882f2" containerName="payload-pre-processing" containerID="cri-o://d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d" gracePeriod=30
Sep 09 20:48:01.790921 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:01.790886    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:01.898149 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:01.898114    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:01.964433 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:01.964409    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:48:01.974491 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:01.974466    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:02.024733 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.024703    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"]
Sep 09 20:48:02.025021 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.024972    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy" containerID="cri-o://ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9" gracePeriod=30
Sep 09 20:48:02.061096 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.061063    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:02.068115 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.068095    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l9kr\" (UniqueName: \"kubernetes.io/projected/e56604e7-403a-4d12-a8ae-930c7c9882f2-kube-api-access-7l9kr\") pod \"e56604e7-403a-4d12-a8ae-930c7c9882f2\" (UID: \"e56604e7-403a-4d12-a8ae-930c7c9882f2\") "
Sep 09 20:48:02.068248 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.068236    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e56604e7-403a-4d12-a8ae-930c7c9882f2-plugins-config-volume\") pod \"e56604e7-403a-4d12-a8ae-930c7c9882f2\" (UID: \"e56604e7-403a-4d12-a8ae-930c7c9882f2\") "
Sep 09 20:48:02.068587 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.068563    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e56604e7-403a-4d12-a8ae-930c7c9882f2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e56604e7-403a-4d12-a8ae-930c7c9882f2" (UID: "e56604e7-403a-4d12-a8ae-930c7c9882f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:48:02.070038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.070017    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e56604e7-403a-4d12-a8ae-930c7c9882f2-kube-api-access-7l9kr" (OuterVolumeSpecName: "kube-api-access-7l9kr") pod "e56604e7-403a-4d12-a8ae-930c7c9882f2" (UID: "e56604e7-403a-4d12-a8ae-930c7c9882f2"). InnerVolumeSpecName "kube-api-access-7l9kr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:48:02.169525 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.169494    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l9kr\" (UniqueName: \"kubernetes.io/projected/e56604e7-403a-4d12-a8ae-930c7c9882f2-kube-api-access-7l9kr\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:02.169525 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.169522    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e56604e7-403a-4d12-a8ae-930c7c9882f2-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:02.479683 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.479648    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:02.664233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.664114    2550 generic.go:356] "Generic (PLEG): container finished" podID="e56604e7-403a-4d12-a8ae-930c7c9882f2" containerID="d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d" exitCode=0
Sep 09 20:48:02.664233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.664205    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"
Sep 09 20:48:02.664233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.664204    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv" event={"ID":"e56604e7-403a-4d12-a8ae-930c7c9882f2","Type":"ContainerDied","Data":"d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d"}
Sep 09 20:48:02.664468 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.664263    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv" event={"ID":"e56604e7-403a-4d12-a8ae-930c7c9882f2","Type":"ContainerDied","Data":"1eb6ea113f0124dbda42e7df1b74503009735ae3949b3ab6cbc9044243e23084"}
Sep 09 20:48:02.664468 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.664286    2550 scope.go:117] "RemoveContainer" containerID="d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d"
Sep 09 20:48:02.673510 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.673489    2550 scope.go:117] "RemoveContainer" containerID="d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d"
Sep 09 20:48:02.673789 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:48:02.673771    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d\": container with ID starting with d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d not found: ID does not exist" containerID="d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d"
Sep 09 20:48:02.673869 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.673804    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d"} err="failed to get container status \"d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d\": rpc error: code = NotFound desc = could not find container \"d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d\": container with ID starting with d8b1d8a52685cd62961096884eaf250bcb59dd9d91e5027191fb316f4bbf653d not found: ID does not exist"
Sep 09 20:48:02.713863 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.713832    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"]
Sep 09 20:48:02.719775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.719749    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5333ad9e-6455dd8598-94qkv"]
Sep 09 20:48:02.842420 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.842390    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"]
Sep 09 20:48:02.842883 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.842862    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e56604e7-403a-4d12-a8ae-930c7c9882f2" containerName="payload-pre-processing"
Sep 09 20:48:02.842883 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.842884    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e56604e7-403a-4d12-a8ae-930c7c9882f2" containerName="payload-pre-processing"
Sep 09 20:48:02.843041 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.843008    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e56604e7-403a-4d12-a8ae-930c7c9882f2" containerName="payload-pre-processing"
Sep 09 20:48:02.843041 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.843025    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="2088dac5-9bab-4fbe-916b-5f1543a87333" containerName="istio-proxy"
Sep 09 20:48:02.845640 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.845619    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:02.848370 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.848351    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fd5982ad-openshift-default-dockercfg-ghd6g\""
Sep 09 20:48:02.862914 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.862895    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"]
Sep 09 20:48:02.883851 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.883816    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:02.976249 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.976214    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-socket\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:02.976426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.976255    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03a47990-a5f8-47fd-b069-96310a8c1209-istiod-ca-cert\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:02.976426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.976323    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-data\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:02.976426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.976359    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03a47990-a5f8-47fd-b069-96310a8c1209-istio-podinfo\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:02.976426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.976385    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-certs\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:02.976426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.976399    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-istio-token\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:02.976594 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.976428    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-credential-socket\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:02.976594 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.976467    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-envoy\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:02.976594 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:02.976532    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7rb2\" (UniqueName: \"kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-kube-api-access-g7rb2\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.018951 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.018919    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:03.078014 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.077982    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-socket\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078014 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078018    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03a47990-a5f8-47fd-b069-96310a8c1209-istiod-ca-cert\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078045    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-data\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078065    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03a47990-a5f8-47fd-b069-96310a8c1209-istio-podinfo\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078091    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-certs\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078115    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-istio-token\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078154    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-credential-socket\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078236    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-envoy\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078604 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078337    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7rb2\" (UniqueName: \"kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-kube-api-access-g7rb2\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078604 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078424    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-socket\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078604 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078491    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-data\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078604 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078573    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-certs\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.078765 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078680    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-credential-socket\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.079003 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.078983    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03a47990-a5f8-47fd-b069-96310a8c1209-istiod-ca-cert\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.080498 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.080476    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-envoy\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.080656 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.080639    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03a47990-a5f8-47fd-b069-96310a8c1209-istio-podinfo\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.088026 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.088005    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-istio-token\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.088143 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.088051    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7rb2\" (UniqueName: \"kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-kube-api-access-g7rb2\") pod \"e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") " pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.155627 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.155592    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:03.430065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.430039    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"]
Sep 09 20:48:03.431133 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:48:03.431103    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a47990_a5f8_47fd_b069_96310a8c1209.slice/crio-cde949af586708b87bcf5617b0873b83924415d7f5ce1cee52b307433339b9df WatchSource:0}: Error finding container cde949af586708b87bcf5617b0873b83924415d7f5ce1cee52b307433339b9df: Status 404 returned error can't find the container with id cde949af586708b87bcf5617b0873b83924415d7f5ce1cee52b307433339b9df
Sep 09 20:48:03.434327 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.434283    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:48:03.434426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.434372    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:48:03.434426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.434420    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:48:03.673427 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.673389    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" event={"ID":"03a47990-a5f8-47fd-b069-96310a8c1209","Type":"ContainerStarted","Data":"59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd"}
Sep 09 20:48:03.673621 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.673449    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" event={"ID":"03a47990-a5f8-47fd-b069-96310a8c1209","Type":"ContainerStarted","Data":"cde949af586708b87bcf5617b0873b83924415d7f5ce1cee52b307433339b9df"}
Sep 09 20:48:03.692002 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:03.691905    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" podStartSLOduration=1.691889052 podStartE2EDuration="1.691889052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:03.690873735 +0000 UTC m=+943.841546105" watchObservedRunningTime="2026-09-09 20:48:03.691889052 +0000 UTC m=+943.842561394"
Sep 09 20:48:04.156256 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:04.156223    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:04.161881 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:04.161858    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:04.280012 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:04.279979    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e56604e7-403a-4d12-a8ae-930c7c9882f2" path="/var/lib/kubelet/pods/e56604e7-403a-4d12-a8ae-930c7c9882f2/volumes"
Sep 09 20:48:04.681076 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:04.681039    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:04.686707 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:04.686683    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:05.047951 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:05.047914    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:05.138396 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:05.138364    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:06.218876 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:06.218838    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:06.316499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:06.316464    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:06.549194 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:06.549080    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:06.711588 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:06.711555    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:06.811096 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:06.811004    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:06.897022 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:06.896989    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"]
Sep 09 20:48:06.900197 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:06.900156    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:06.902323 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:06.902299    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-fd5982ad\""
Sep 09 20:48:06.902452 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:06.902429    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-fd5982ad-dockercfg-rbdh7\""
Sep 09 20:48:06.912240 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:06.912219    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"]
Sep 09 20:48:07.017849 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.017813    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6bn9\" (UniqueName: \"kubernetes.io/projected/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-kube-api-access-t6bn9\") pod \"payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk\" (UID: \"122ccfcf-b1cd-42fb-9e43-81a90205cdf6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:07.018024 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.017901    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk\" (UID: \"122ccfcf-b1cd-42fb-9e43-81a90205cdf6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:07.118504 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.118437    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk\" (UID: \"122ccfcf-b1cd-42fb-9e43-81a90205cdf6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:07.118636 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.118526    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6bn9\" (UniqueName: \"kubernetes.io/projected/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-kube-api-access-t6bn9\") pod \"payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk\" (UID: \"122ccfcf-b1cd-42fb-9e43-81a90205cdf6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:07.119134 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.119114    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk\" (UID: \"122ccfcf-b1cd-42fb-9e43-81a90205cdf6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:07.126668 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.126650    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6bn9\" (UniqueName: \"kubernetes.io/projected/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-kube-api-access-t6bn9\") pod \"payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk\" (UID: \"122ccfcf-b1cd-42fb-9e43-81a90205cdf6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:07.211512 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.211478    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:07.291884 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.291816    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:48:07.421846 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.421761    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-credential-socket\") pod \"fd4555ea-212d-4cb2-9cd6-938f120fda04\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") "
Sep 09 20:48:07.421846 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.421831    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-certs\") pod \"fd4555ea-212d-4cb2-9cd6-938f120fda04\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") "
Sep 09 20:48:07.422065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.421883    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-token\") pod \"fd4555ea-212d-4cb2-9cd6-938f120fda04\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") "
Sep 09 20:48:07.422065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.421906    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwrxb\" (UniqueName: \"kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-kube-api-access-dwrxb\") pod \"fd4555ea-212d-4cb2-9cd6-938f120fda04\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") "
Sep 09 20:48:07.422065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.421933    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-envoy\") pod \"fd4555ea-212d-4cb2-9cd6-938f120fda04\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") "
Sep 09 20:48:07.422065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.421965    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-podinfo\") pod \"fd4555ea-212d-4cb2-9cd6-938f120fda04\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") "
Sep 09 20:48:07.422065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.421974    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fd4555ea-212d-4cb2-9cd6-938f120fda04" (UID: "fd4555ea-212d-4cb2-9cd6-938f120fda04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:48:07.422065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.421989    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-data\") pod \"fd4555ea-212d-4cb2-9cd6-938f120fda04\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") "
Sep 09 20:48:07.422065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422055    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd4555ea-212d-4cb2-9cd6-938f120fda04-istiod-ca-cert\") pod \"fd4555ea-212d-4cb2-9cd6-938f120fda04\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") "
Sep 09 20:48:07.422463 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422130    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-socket\") pod \"fd4555ea-212d-4cb2-9cd6-938f120fda04\" (UID: \"fd4555ea-212d-4cb2-9cd6-938f120fda04\") "
Sep 09 20:48:07.422463 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422204    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fd4555ea-212d-4cb2-9cd6-938f120fda04" (UID: "fd4555ea-212d-4cb2-9cd6-938f120fda04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:48:07.422463 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422285    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-data" (OuterVolumeSpecName: "istio-data") pod "fd4555ea-212d-4cb2-9cd6-938f120fda04" (UID: "fd4555ea-212d-4cb2-9cd6-938f120fda04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:48:07.422463 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422448    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fd4555ea-212d-4cb2-9cd6-938f120fda04" (UID: "fd4555ea-212d-4cb2-9cd6-938f120fda04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:48:07.422635 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422497    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd4555ea-212d-4cb2-9cd6-938f120fda04-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fd4555ea-212d-4cb2-9cd6-938f120fda04" (UID: "fd4555ea-212d-4cb2-9cd6-938f120fda04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:48:07.422837 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422814    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:07.422934 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422852    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:07.422934 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422875    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd4555ea-212d-4cb2-9cd6-938f120fda04-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:07.422934 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422889    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:07.422934 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.422903    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:07.424333 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.424298    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd4555ea-212d-4cb2-9cd6-938f120fda04" (UID: "fd4555ea-212d-4cb2-9cd6-938f120fda04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:48:07.424553 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.424534    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fd4555ea-212d-4cb2-9cd6-938f120fda04" (UID: "fd4555ea-212d-4cb2-9cd6-938f120fda04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:48:07.424728 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.424701    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-kube-api-access-dwrxb" (OuterVolumeSpecName: "kube-api-access-dwrxb") pod "fd4555ea-212d-4cb2-9cd6-938f120fda04" (UID: "fd4555ea-212d-4cb2-9cd6-938f120fda04"). InnerVolumeSpecName "kube-api-access-dwrxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:48:07.424728 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.424718    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fd4555ea-212d-4cb2-9cd6-938f120fda04" (UID: "fd4555ea-212d-4cb2-9cd6-938f120fda04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:48:07.494989 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.494959    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"]
Sep 09 20:48:07.495739 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:48:07.495713    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod122ccfcf_b1cd_42fb_9e43_81a90205cdf6.slice/crio-5efb299ab66a711a23a13ea92225b1a9839aa37f07a0f38ec67b49a43a2403d6 WatchSource:0}: Error finding container 5efb299ab66a711a23a13ea92225b1a9839aa37f07a0f38ec67b49a43a2403d6: Status 404 returned error can't find the container with id 5efb299ab66a711a23a13ea92225b1a9839aa37f07a0f38ec67b49a43a2403d6
Sep 09 20:48:07.524289 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.524270    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:07.524365 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.524308    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwrxb\" (UniqueName: \"kubernetes.io/projected/fd4555ea-212d-4cb2-9cd6-938f120fda04-kube-api-access-dwrxb\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:07.524365 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.524320    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:07.524365 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.524329    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd4555ea-212d-4cb2-9cd6-938f120fda04-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:07.699270 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.699155    2550 generic.go:356] "Generic (PLEG): container finished" podID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerID="ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9" exitCode=0
Sep 09 20:48:07.699270 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.699224    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" event={"ID":"fd4555ea-212d-4cb2-9cd6-938f120fda04","Type":"ContainerDied","Data":"ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9"}
Sep 09 20:48:07.699461 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.699235    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"
Sep 09 20:48:07.699461 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.699284    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps" event={"ID":"fd4555ea-212d-4cb2-9cd6-938f120fda04","Type":"ContainerDied","Data":"32225cd5bc1874de32ab6c1448b5de3a195e0af376660de0b958e6a6ced77abe"}
Sep 09 20:48:07.699461 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.699308    2550 scope.go:117] "RemoveContainer" containerID="ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9"
Sep 09 20:48:07.701139 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.701114    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk" event={"ID":"122ccfcf-b1cd-42fb-9e43-81a90205cdf6","Type":"ContainerStarted","Data":"d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4"}
Sep 09 20:48:07.701294 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.701181    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk" event={"ID":"122ccfcf-b1cd-42fb-9e43-81a90205cdf6","Type":"ContainerStarted","Data":"5efb299ab66a711a23a13ea92225b1a9839aa37f07a0f38ec67b49a43a2403d6"}
Sep 09 20:48:07.701294 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.701215    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:07.708580 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.708564    2550 scope.go:117] "RemoveContainer" containerID="c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e"
Sep 09 20:48:07.716930 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.716915    2550 scope.go:117] "RemoveContainer" containerID="ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9"
Sep 09 20:48:07.717244 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:48:07.717153    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9\": container with ID starting with ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9 not found: ID does not exist" containerID="ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9"
Sep 09 20:48:07.717296 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.717253    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9"} err="failed to get container status \"ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9\": rpc error: code = NotFound desc = could not find container \"ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9\": container with ID starting with ea8dd224fc61a2d649a5be6e1d72e087bad18402e01cd92d08c24cd0e04772a9 not found: ID does not exist"
Sep 09 20:48:07.717296 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.717272    2550 scope.go:117] "RemoveContainer" containerID="c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e"
Sep 09 20:48:07.717512 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:48:07.717493    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e\": container with ID starting with c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e not found: ID does not exist" containerID="c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e"
Sep 09 20:48:07.717553 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.717517    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e"} err="failed to get container status \"c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e\": rpc error: code = NotFound desc = could not find container \"c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e\": container with ID starting with c47e0d52ae936e218c050ff4e62d166651431397fa4437979cb6efeb268fb94e not found: ID does not exist"
Sep 09 20:48:07.721127 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.721088    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk" podStartSLOduration=1.721075619 podStartE2EDuration="1.721075619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:07.719670109 +0000 UTC m=+947.870342451" watchObservedRunningTime="2026-09-09 20:48:07.721075619 +0000 UTC m=+947.871747963"
Sep 09 20:48:07.739490 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.739467    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"]
Sep 09 20:48:07.747972 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:07.747941    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-5333ad9e-gw-openshift-default-58cbbc958d-pqsps"]
Sep 09 20:48:08.281418 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:08.281382    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" path="/var/lib/kubelet/pods/fd4555ea-212d-4cb2-9cd6-938f120fda04/volumes"
Sep 09 20:48:10.380963 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.380928    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"]
Sep 09 20:48:10.381344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.381330    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy"
Sep 09 20:48:10.381344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.381345    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy"
Sep 09 20:48:10.381430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.381363    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy"
Sep 09 20:48:10.381430 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.381369    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy"
Sep 09 20:48:10.381497 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.381444    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy"
Sep 09 20:48:10.381497 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.381454    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd4555ea-212d-4cb2-9cd6-938f120fda04" containerName="istio-proxy"
Sep 09 20:48:10.384112 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.384096    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.386731 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.386710    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2b2343\"/\"default-dockercfg-v4nrn\""
Sep 09 20:48:10.386858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.386743    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2b2343\"/\"openshift-service-ca.crt\""
Sep 09 20:48:10.386858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.386753    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2b2343\"/\"simulator-w0-2b2343-kserve-self-signed-certs\""
Sep 09 20:48:10.386858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.386830    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2b2343\"/\"kube-root-ca.crt\""
Sep 09 20:48:10.397057 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.397034    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"]
Sep 09 20:48:10.554221 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.554158    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-dshm\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.554424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.554265    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-model-cache\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.554424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.554293    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-tmp-dir\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.554424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.554313    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz4d2\" (UniqueName: \"kubernetes.io/projected/5e45b1a2-815c-476d-980e-c505c810c987-kube-api-access-xz4d2\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.554424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.554334    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e45b1a2-815c-476d-980e-c505c810c987-tls-certs\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.554424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.554413    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-home\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.655279 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.655211    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-model-cache\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.655279 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.655242    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-tmp-dir\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.655279 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.655261    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz4d2\" (UniqueName: \"kubernetes.io/projected/5e45b1a2-815c-476d-980e-c505c810c987-kube-api-access-xz4d2\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.655547 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.655284    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e45b1a2-815c-476d-980e-c505c810c987-tls-certs\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.655547 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.655326    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-home\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.655547 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.655407    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-dshm\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.655718 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.655658    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-tmp-dir\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.655718 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.655679    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-model-cache\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.655808 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.655736    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-home\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.657711 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.657690    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-dshm\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.657880 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.657863    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e45b1a2-815c-476d-980e-c505c810c987-tls-certs\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.663409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.663387    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz4d2\" (UniqueName: \"kubernetes.io/projected/5e45b1a2-815c-476d-980e-c505c810c987-kube-api-access-xz4d2\") pod \"simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") " pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.693099 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.693077    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:10.790697 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.790668    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:10.975979 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:10.975952    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"]
Sep 09 20:48:10.977485 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:48:10.977456    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e45b1a2_815c_476d_980e_c505c810c987.slice/crio-301f8d6283e929637ac41096f767901ad4e26026a8f54ac9a2b208aa4d1cce8e WatchSource:0}: Error finding container 301f8d6283e929637ac41096f767901ad4e26026a8f54ac9a2b208aa4d1cce8e: Status 404 returned error can't find the container with id 301f8d6283e929637ac41096f767901ad4e26026a8f54ac9a2b208aa4d1cce8e
Sep 09 20:48:11.727777 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:11.727743    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4" event={"ID":"5e45b1a2-815c-476d-980e-c505c810c987","Type":"ContainerStarted","Data":"da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154"}
Sep 09 20:48:11.728121 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:11.727792    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4" event={"ID":"5e45b1a2-815c-476d-980e-c505c810c987","Type":"ContainerStarted","Data":"301f8d6283e929637ac41096f767901ad4e26026a8f54ac9a2b208aa4d1cce8e"}
Sep 09 20:48:11.757864 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:11.757818    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4" podStartSLOduration=1.757802004 podStartE2EDuration="1.757802004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:11.75591288 +0000 UTC m=+951.906585224" watchObservedRunningTime="2026-09-09 20:48:11.757802004 +0000 UTC m=+951.908474347"
Sep 09 20:48:12.566264 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:12.566233    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:18.710531 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:18.710496    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:20.694205 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:20.694076    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:20.694205 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:20.694146    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:20.706589 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:20.706563    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:20.720284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:20.720254    2550 patch_prober.go:28] interesting pod/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 20:48:20.720412 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:20.720298    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 20:48:20.804823 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:20.804794    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:48:21.797060 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:21.797018    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:22.807715 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:22.807685    2550 generic.go:356] "Generic (PLEG): container finished" podID="03a47990-a5f8-47fd-b069-96310a8c1209" containerID="59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd" exitCode=0
Sep 09 20:48:22.808107 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:22.807759    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" event={"ID":"03a47990-a5f8-47fd-b069-96310a8c1209","Type":"ContainerDied","Data":"59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd"}
Sep 09 20:48:22.808175 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:22.808149    2550 scope.go:117] "RemoveContainer" containerID="59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd"
Sep 09 20:48:22.809361 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:22.809336    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:48:22.809439 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:22.809403    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:48:22.809477 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:22.809438    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:48:23.815873 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:23.815835    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" event={"ID":"03a47990-a5f8-47fd-b069-96310a8c1209","Type":"ContainerStarted","Data":"4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc"}
Sep 09 20:48:24.160257 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:24.156772    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:24.160257 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:24.156905    2550 patch_prober.go:28] interesting pod/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 09 20:48:24.160257 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:24.156949    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 09 20:48:24.220891 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:24.220853    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:25.156155 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:25.156114    2550 patch_prober.go:28] interesting pod/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 09 20:48:25.156736 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:25.156703    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 09 20:48:26.156527 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:26.156494    2550 patch_prober.go:28] interesting pod/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 09 20:48:26.157095 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:26.157064    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 09 20:48:27.156687 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:27.156642    2550 patch_prober.go:28] interesting pod/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 09 20:48:27.157199 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:27.156730    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 09 20:48:28.156567 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:28.156526    2550 patch_prober.go:28] interesting pod/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 09 20:48:28.156869 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:28.156841    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 09 20:48:29.156348 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:29.156305    2550 patch_prober.go:28] interesting pod/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 09 20:48:29.156538 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:29.156395    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 09 20:48:30.160036 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:30.160005    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:30.160513 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:30.160397    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:30.161064 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:30.161042    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:36.876932 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:36.876895    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:36.906054 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:36.906014    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"]
Sep 09 20:48:36.906658 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:36.906627    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk" podUID="122ccfcf-b1cd-42fb-9e43-81a90205cdf6" containerName="payload-pre-processing" containerID="cri-o://d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4" gracePeriod=30
Sep 09 20:48:36.979562 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:36.979522    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:37.078509 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.077715    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:37.164710 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.164684    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:37.165096 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.165068    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:37.214233 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.214207    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6bn9\" (UniqueName: \"kubernetes.io/projected/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-kube-api-access-t6bn9\") pod \"122ccfcf-b1cd-42fb-9e43-81a90205cdf6\" (UID: \"122ccfcf-b1cd-42fb-9e43-81a90205cdf6\") "
Sep 09 20:48:37.214350 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.214263    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-plugins-config-volume\") pod \"122ccfcf-b1cd-42fb-9e43-81a90205cdf6\" (UID: \"122ccfcf-b1cd-42fb-9e43-81a90205cdf6\") "
Sep 09 20:48:37.214667 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.214639    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "122ccfcf-b1cd-42fb-9e43-81a90205cdf6" (UID: "122ccfcf-b1cd-42fb-9e43-81a90205cdf6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:48:37.216208 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.216185    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-kube-api-access-t6bn9" (OuterVolumeSpecName: "kube-api-access-t6bn9") pod "122ccfcf-b1cd-42fb-9e43-81a90205cdf6" (UID: "122ccfcf-b1cd-42fb-9e43-81a90205cdf6"). InnerVolumeSpecName "kube-api-access-t6bn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:48:37.315282 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.315240    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6bn9\" (UniqueName: \"kubernetes.io/projected/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-kube-api-access-t6bn9\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:37.315282 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.315278    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/122ccfcf-b1cd-42fb-9e43-81a90205cdf6-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:37.904129 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.904091    2550 generic.go:356] "Generic (PLEG): container finished" podID="122ccfcf-b1cd-42fb-9e43-81a90205cdf6" containerID="d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4" exitCode=0
Sep 09 20:48:37.904129 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.904132    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk" event={"ID":"122ccfcf-b1cd-42fb-9e43-81a90205cdf6","Type":"ContainerDied","Data":"d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4"}
Sep 09 20:48:37.904576 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.904139    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"
Sep 09 20:48:37.904576 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.904190    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk" event={"ID":"122ccfcf-b1cd-42fb-9e43-81a90205cdf6","Type":"ContainerDied","Data":"5efb299ab66a711a23a13ea92225b1a9839aa37f07a0f38ec67b49a43a2403d6"}
Sep 09 20:48:37.904576 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.904218    2550 scope.go:117] "RemoveContainer" containerID="d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4"
Sep 09 20:48:37.913639 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.913434    2550 scope.go:117] "RemoveContainer" containerID="d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4"
Sep 09 20:48:37.913716 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:48:37.913704    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4\": container with ID starting with d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4 not found: ID does not exist" containerID="d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4"
Sep 09 20:48:37.913755 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.913727    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4"} err="failed to get container status \"d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4\": rpc error: code = NotFound desc = could not find container \"d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4\": container with ID starting with d2d91c2fd19b6931429654d6a6d3451f7b0c218490ff588f19f0ad88dd0210c4 not found: ID does not exist"
Sep 09 20:48:37.925964 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.925939    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"]
Sep 09 20:48:37.930315 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:37.930286    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd5982ad-68884c9878-89kgk"]
Sep 09 20:48:38.203600 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:38.203510    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"]
Sep 09 20:48:38.203763 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:38.203739    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy" containerID="cri-o://4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc" gracePeriod=30
Sep 09 20:48:38.228705 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:38.228670    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:38.286327 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:38.286298    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="122ccfcf-b1cd-42fb-9e43-81a90205cdf6" path="/var/lib/kubelet/pods/122ccfcf-b1cd-42fb-9e43-81a90205cdf6/volumes"
Sep 09 20:48:38.647836 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:38.647798    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:38.959381 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:38.959338    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:39.060203 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:39.060157    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:41.770104 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:41.770070    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:41.871368 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:41.871330    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:43.053088 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.053012    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:43.155544 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.155504    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:43.254033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.253976    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:43.462599 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.462577    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:43.570008 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.569924    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:43.578512 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578480    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-socket\") pod \"03a47990-a5f8-47fd-b069-96310a8c1209\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") "
Sep 09 20:48:43.578690 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578527    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-envoy\") pod \"03a47990-a5f8-47fd-b069-96310a8c1209\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") "
Sep 09 20:48:43.578690 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578611    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-data\") pod \"03a47990-a5f8-47fd-b069-96310a8c1209\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") "
Sep 09 20:48:43.578690 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578639    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-certs\") pod \"03a47990-a5f8-47fd-b069-96310a8c1209\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") "
Sep 09 20:48:43.578874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578718    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7rb2\" (UniqueName: \"kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-kube-api-access-g7rb2\") pod \"03a47990-a5f8-47fd-b069-96310a8c1209\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") "
Sep 09 20:48:43.578874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578768    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03a47990-a5f8-47fd-b069-96310a8c1209-istiod-ca-cert\") pod \"03a47990-a5f8-47fd-b069-96310a8c1209\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") "
Sep 09 20:48:43.578874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578802    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03a47990-a5f8-47fd-b069-96310a8c1209-istio-podinfo\") pod \"03a47990-a5f8-47fd-b069-96310a8c1209\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") "
Sep 09 20:48:43.578874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578832    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-istio-token\") pod \"03a47990-a5f8-47fd-b069-96310a8c1209\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") "
Sep 09 20:48:43.578874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578873    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-credential-socket\") pod \"03a47990-a5f8-47fd-b069-96310a8c1209\" (UID: \"03a47990-a5f8-47fd-b069-96310a8c1209\") "
Sep 09 20:48:43.579136 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578756    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "03a47990-a5f8-47fd-b069-96310a8c1209" (UID: "03a47990-a5f8-47fd-b069-96310a8c1209"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:48:43.579136 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.579093    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03a47990-a5f8-47fd-b069-96310a8c1209-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "03a47990-a5f8-47fd-b069-96310a8c1209" (UID: "03a47990-a5f8-47fd-b069-96310a8c1209"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:48:43.579284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.579135    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "03a47990-a5f8-47fd-b069-96310a8c1209" (UID: "03a47990-a5f8-47fd-b069-96310a8c1209"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:48:43.579284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.579233    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03a47990-a5f8-47fd-b069-96310a8c1209-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:43.579284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.579251    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:43.579284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.579267    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:43.579284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578851    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-data" (OuterVolumeSpecName: "istio-data") pod "03a47990-a5f8-47fd-b069-96310a8c1209" (UID: "03a47990-a5f8-47fd-b069-96310a8c1209"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:48:43.579531 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.578898    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "03a47990-a5f8-47fd-b069-96310a8c1209" (UID: "03a47990-a5f8-47fd-b069-96310a8c1209"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:48:43.581308 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.581272    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03a47990-a5f8-47fd-b069-96310a8c1209-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "03a47990-a5f8-47fd-b069-96310a8c1209" (UID: "03a47990-a5f8-47fd-b069-96310a8c1209"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:48:43.581944 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.581913    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-kube-api-access-g7rb2" (OuterVolumeSpecName: "kube-api-access-g7rb2") pod "03a47990-a5f8-47fd-b069-96310a8c1209" (UID: "03a47990-a5f8-47fd-b069-96310a8c1209"). InnerVolumeSpecName "kube-api-access-g7rb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:48:43.582250 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.582227    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "03a47990-a5f8-47fd-b069-96310a8c1209" (UID: "03a47990-a5f8-47fd-b069-96310a8c1209"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:48:43.582340 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.582281    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-istio-token" (OuterVolumeSpecName: "istio-token") pod "03a47990-a5f8-47fd-b069-96310a8c1209" (UID: "03a47990-a5f8-47fd-b069-96310a8c1209"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:48:43.669796 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.669760    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:43.680341 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.680316    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:43.680341 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.680339    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:43.680518 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.680349    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7rb2\" (UniqueName: \"kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-kube-api-access-g7rb2\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:43.680518 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.680359    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03a47990-a5f8-47fd-b069-96310a8c1209-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:43.680518 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.680368    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03a47990-a5f8-47fd-b069-96310a8c1209-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:43.680518 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.680377    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03a47990-a5f8-47fd-b069-96310a8c1209-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:48:43.683925 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.683900    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"]
Sep 09 20:48:43.684303 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.684291    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy"
Sep 09 20:48:43.684350 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.684307    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy"
Sep 09 20:48:43.684350 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.684323    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy"
Sep 09 20:48:43.684350 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.684329    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy"
Sep 09 20:48:43.684350 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.684347    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="122ccfcf-b1cd-42fb-9e43-81a90205cdf6" containerName="payload-pre-processing"
Sep 09 20:48:43.684517 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.684352    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="122ccfcf-b1cd-42fb-9e43-81a90205cdf6" containerName="payload-pre-processing"
Sep 09 20:48:43.684517 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.684412    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy"
Sep 09 20:48:43.684517 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.684419    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="122ccfcf-b1cd-42fb-9e43-81a90205cdf6" containerName="payload-pre-processing"
Sep 09 20:48:43.684517 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.684426    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" containerName="istio-proxy"
Sep 09 20:48:43.687121 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.687104    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:48:43.690152 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.689672    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-3d3c9036\""
Sep 09 20:48:43.690152 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.689760    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-3d3c9036-dockercfg-f82j7\""
Sep 09 20:48:43.701669 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.701639    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"]
Sep 09 20:48:43.781214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.781181    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985\" (UID: \"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:48:43.781367 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.781285    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5t8q\" (UniqueName: \"kubernetes.io/projected/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-kube-api-access-t5t8q\") pod \"payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985\" (UID: \"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:48:43.817719 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.817686    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"]
Sep 09 20:48:43.821297 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.821231    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:48:43.831867 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.831836    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"]
Sep 09 20:48:43.882252 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.882218    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5t8q\" (UniqueName: \"kubernetes.io/projected/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-kube-api-access-t5t8q\") pod \"payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985\" (UID: \"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:48:43.882416 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.882258    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70965178-adfb-4487-9673-9b6ffe68ad8a-plugins-config-volume\") pod \"payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27\" (UID: \"70965178-adfb-4487-9673-9b6ffe68ad8a\") " pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:48:43.882416 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.882285    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx72k\" (UniqueName: \"kubernetes.io/projected/70965178-adfb-4487-9673-9b6ffe68ad8a-kube-api-access-cx72k\") pod \"payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27\" (UID: \"70965178-adfb-4487-9673-9b6ffe68ad8a\") " pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:48:43.882416 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.882358    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985\" (UID: \"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:48:43.883023 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.883002    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985\" (UID: \"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:48:43.890666 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.890642    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5t8q\" (UniqueName: \"kubernetes.io/projected/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-kube-api-access-t5t8q\") pod \"payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985\" (UID: \"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:48:43.940929 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.940891    2550 generic.go:356] "Generic (PLEG): container finished" podID="03a47990-a5f8-47fd-b069-96310a8c1209" containerID="4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc" exitCode=0
Sep 09 20:48:43.941076 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.940963    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"
Sep 09 20:48:43.941076 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.940966    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" event={"ID":"03a47990-a5f8-47fd-b069-96310a8c1209","Type":"ContainerDied","Data":"4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc"}
Sep 09 20:48:43.941076 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.941021    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl" event={"ID":"03a47990-a5f8-47fd-b069-96310a8c1209","Type":"ContainerDied","Data":"cde949af586708b87bcf5617b0873b83924415d7f5ce1cee52b307433339b9df"}
Sep 09 20:48:43.941076 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.941044    2550 scope.go:117] "RemoveContainer" containerID="4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc"
Sep 09 20:48:43.950457 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.950427    2550 scope.go:117] "RemoveContainer" containerID="59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd"
Sep 09 20:48:43.958971 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.958951    2550 scope.go:117] "RemoveContainer" containerID="4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc"
Sep 09 20:48:43.959250 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:48:43.959230    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc\": container with ID starting with 4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc not found: ID does not exist" containerID="4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc"
Sep 09 20:48:43.959331 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.959263    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc"} err="failed to get container status \"4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc\": rpc error: code = NotFound desc = could not find container \"4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc\": container with ID starting with 4af4f8c96b8a065ed73ef437a5214f81925a3feddcb898e96a4d8d05667060dc not found: ID does not exist"
Sep 09 20:48:43.959331 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.959287    2550 scope.go:117] "RemoveContainer" containerID="59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd"
Sep 09 20:48:43.959552 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:48:43.959533    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd\": container with ID starting with 59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd not found: ID does not exist" containerID="59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd"
Sep 09 20:48:43.959592 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.959558    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd"} err="failed to get container status \"59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd\": rpc error: code = NotFound desc = could not find container \"59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd\": container with ID starting with 59ebb62376af6fae6a8aaa5a71a83c3b87b768ebe141ecbd3058e3a7d9aa60dd not found: ID does not exist"
Sep 09 20:48:43.965715 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.965694    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"]
Sep 09 20:48:43.970589 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.970566    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fd5982ad-openshift-default-568c668cfd-6mwkl"]
Sep 09 20:48:43.983395 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.983375    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70965178-adfb-4487-9673-9b6ffe68ad8a-plugins-config-volume\") pod \"payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27\" (UID: \"70965178-adfb-4487-9673-9b6ffe68ad8a\") " pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:48:43.983486 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.983405    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx72k\" (UniqueName: \"kubernetes.io/projected/70965178-adfb-4487-9673-9b6ffe68ad8a-kube-api-access-cx72k\") pod \"payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27\" (UID: \"70965178-adfb-4487-9673-9b6ffe68ad8a\") " pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:48:43.984011 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.983995    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70965178-adfb-4487-9673-9b6ffe68ad8a-plugins-config-volume\") pod \"payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27\" (UID: \"70965178-adfb-4487-9673-9b6ffe68ad8a\") " pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:48:43.991239 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:43.991217    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx72k\" (UniqueName: \"kubernetes.io/projected/70965178-adfb-4487-9673-9b6ffe68ad8a-kube-api-access-cx72k\") pod \"payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27\" (UID: \"70965178-adfb-4487-9673-9b6ffe68ad8a\") " pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:48:44.004972 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.004937    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:48:44.133882 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.133788    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:48:44.282764 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:48:44.282734    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cd5820b_8e4c_4153_bc9b_cf5fafa22ef7.slice/crio-41d15a3a832840bcf6ca576153f8d32ce7e3db2e11fc07caf1589e6a2d7592dc WatchSource:0}: Error finding container 41d15a3a832840bcf6ca576153f8d32ce7e3db2e11fc07caf1589e6a2d7592dc: Status 404 returned error can't find the container with id 41d15a3a832840bcf6ca576153f8d32ce7e3db2e11fc07caf1589e6a2d7592dc
Sep 09 20:48:44.285025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.285000    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03a47990-a5f8-47fd-b069-96310a8c1209" path="/var/lib/kubelet/pods/03a47990-a5f8-47fd-b069-96310a8c1209/volumes"
Sep 09 20:48:44.285367 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.285350    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"]
Sep 09 20:48:44.407854 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.407817    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"]
Sep 09 20:48:44.409398 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:48:44.409369    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70965178_adfb_4487_9673_9b6ffe68ad8a.slice/crio-e5a7a360bd3a25bc8e97c18739093f0bb3f0a1398970a63eb68d413b9209144f WatchSource:0}: Error finding container e5a7a360bd3a25bc8e97c18739093f0bb3f0a1398970a63eb68d413b9209144f: Status 404 returned error can't find the container with id e5a7a360bd3a25bc8e97c18739093f0bb3f0a1398970a63eb68d413b9209144f
Sep 09 20:48:44.950201 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.950037    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985" event={"ID":"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7","Type":"ContainerStarted","Data":"00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd"}
Sep 09 20:48:44.950201 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.950113    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:48:44.950201 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.950137    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985" event={"ID":"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7","Type":"ContainerStarted","Data":"41d15a3a832840bcf6ca576153f8d32ce7e3db2e11fc07caf1589e6a2d7592dc"}
Sep 09 20:48:44.953306 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.953276    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27" event={"ID":"70965178-adfb-4487-9673-9b6ffe68ad8a","Type":"ContainerStarted","Data":"1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df"}
Sep 09 20:48:44.953465 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.953329    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27" event={"ID":"70965178-adfb-4487-9673-9b6ffe68ad8a","Type":"ContainerStarted","Data":"e5a7a360bd3a25bc8e97c18739093f0bb3f0a1398970a63eb68d413b9209144f"}
Sep 09 20:48:44.953540 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.953520    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:48:44.966311 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.966253    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985" podStartSLOduration=1.966237928 podStartE2EDuration="1.966237928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:44.965357306 +0000 UTC m=+985.116029651" watchObservedRunningTime="2026-09-09 20:48:44.966237928 +0000 UTC m=+985.116910272"
Sep 09 20:48:44.983480 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:44.983431    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27" podStartSLOduration=1.983416601 podStartE2EDuration="1.983416601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:44.982150468 +0000 UTC m=+985.132822812" watchObservedRunningTime="2026-09-09 20:48:44.983416601 +0000 UTC m=+985.134088943"
Sep 09 20:48:55.963053 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:55.963023    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:48:55.963641 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:55.963108    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:48:59.370204 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:48:59.370148    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:05.211820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:05.211785    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:09.354140 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:09.354106    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:10.055144 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:10.055107    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:10.140855 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:10.140817    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:10.212039 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:10.211953    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:10.284369 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:10.284333    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:10.368434 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:10.368318    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:10.657435 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:10.657354    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:13.116075 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:13.116046    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:13.734962 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:13.734928    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:13.767981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:13.767949    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"]
Sep 09 20:49:13.768260 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:13.768235    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27" podUID="70965178-adfb-4487-9673-9b6ffe68ad8a" containerName="payload-processing" containerID="cri-o://1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df" gracePeriod=30
Sep 09 20:49:13.774364 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:13.774137    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"]
Sep 09 20:49:13.774498 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:13.774373    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985" podUID="0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7" containerName="payload-pre-processing" containerID="cri-o://00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd" gracePeriod=30
Sep 09 20:49:13.812335 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:13.812305    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:13.883155 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:13.883121    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:13.962136 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:13.962102    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:14.041668 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.041642    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:14.053521 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.053501    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:49:14.056249 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.056232    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:49:14.124686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.124656    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:14.131142 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.131114    2550 generic.go:356] "Generic (PLEG): container finished" podID="70965178-adfb-4487-9673-9b6ffe68ad8a" containerID="1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df" exitCode=0
Sep 09 20:49:14.131336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.131188    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"
Sep 09 20:49:14.131336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.131201    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27" event={"ID":"70965178-adfb-4487-9673-9b6ffe68ad8a","Type":"ContainerDied","Data":"1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df"}
Sep 09 20:49:14.131336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.131277    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27" event={"ID":"70965178-adfb-4487-9673-9b6ffe68ad8a","Type":"ContainerDied","Data":"e5a7a360bd3a25bc8e97c18739093f0bb3f0a1398970a63eb68d413b9209144f"}
Sep 09 20:49:14.131336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.131321    2550 scope.go:117] "RemoveContainer" containerID="1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df"
Sep 09 20:49:14.133024 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.132991    2550 generic.go:356] "Generic (PLEG): container finished" podID="0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7" containerID="00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd" exitCode=0
Sep 09 20:49:14.133133 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.133043    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"
Sep 09 20:49:14.133133 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.133048    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985" event={"ID":"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7","Type":"ContainerDied","Data":"00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd"}
Sep 09 20:49:14.133133 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.133128    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985" event={"ID":"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7","Type":"ContainerDied","Data":"41d15a3a832840bcf6ca576153f8d32ce7e3db2e11fc07caf1589e6a2d7592dc"}
Sep 09 20:49:14.140707 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.140686    2550 scope.go:117] "RemoveContainer" containerID="1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df"
Sep 09 20:49:14.141023 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:49:14.141001    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df\": container with ID starting with 1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df not found: ID does not exist" containerID="1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df"
Sep 09 20:49:14.141069 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.141032    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df"} err="failed to get container status \"1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df\": rpc error: code = NotFound desc = could not find container \"1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df\": container with ID starting with 1efe7cc4c25e5c110a46f35b59a116040615c50907fda09fda0175fb33e0a0df not found: ID does not exist"
Sep 09 20:49:14.141069 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.141048    2550 scope.go:117] "RemoveContainer" containerID="00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd"
Sep 09 20:49:14.149227 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.149210    2550 scope.go:117] "RemoveContainer" containerID="00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd"
Sep 09 20:49:14.149470 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:49:14.149448    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd\": container with ID starting with 00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd not found: ID does not exist" containerID="00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd"
Sep 09 20:49:14.149561 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.149476    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd"} err="failed to get container status \"00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd\": rpc error: code = NotFound desc = could not find container \"00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd\": container with ID starting with 00c99bd583b2f62085f44d7080cf13084905fc038d22149876ffefe9433784fd not found: ID does not exist"
Sep 09 20:49:14.165794 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.165765    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5t8q\" (UniqueName: \"kubernetes.io/projected/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-kube-api-access-t5t8q\") pod \"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7\" (UID: \"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7\") "
Sep 09 20:49:14.165913 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.165828    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-plugins-config-volume\") pod \"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7\" (UID: \"0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7\") "
Sep 09 20:49:14.165913 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.165863    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx72k\" (UniqueName: \"kubernetes.io/projected/70965178-adfb-4487-9673-9b6ffe68ad8a-kube-api-access-cx72k\") pod \"70965178-adfb-4487-9673-9b6ffe68ad8a\" (UID: \"70965178-adfb-4487-9673-9b6ffe68ad8a\") "
Sep 09 20:49:14.166026 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.165919    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70965178-adfb-4487-9673-9b6ffe68ad8a-plugins-config-volume\") pod \"70965178-adfb-4487-9673-9b6ffe68ad8a\" (UID: \"70965178-adfb-4487-9673-9b6ffe68ad8a\") "
Sep 09 20:49:14.166269 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.166247    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7" (UID: "0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:49:14.166336 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.166304    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70965178-adfb-4487-9673-9b6ffe68ad8a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "70965178-adfb-4487-9673-9b6ffe68ad8a" (UID: "70965178-adfb-4487-9673-9b6ffe68ad8a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:49:14.167753 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.167730    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-kube-api-access-t5t8q" (OuterVolumeSpecName: "kube-api-access-t5t8q") pod "0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7" (UID: "0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7"). InnerVolumeSpecName "kube-api-access-t5t8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:49:14.167961 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.167944    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70965178-adfb-4487-9673-9b6ffe68ad8a-kube-api-access-cx72k" (OuterVolumeSpecName: "kube-api-access-cx72k") pod "70965178-adfb-4487-9673-9b6ffe68ad8a" (UID: "70965178-adfb-4487-9673-9b6ffe68ad8a"). InnerVolumeSpecName "kube-api-access-cx72k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:49:14.267552 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.267513    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70965178-adfb-4487-9673-9b6ffe68ad8a-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:49:14.267552 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.267544    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5t8q\" (UniqueName: \"kubernetes.io/projected/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-kube-api-access-t5t8q\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:49:14.267552 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.267553    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:49:14.267552 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.267562    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx72k\" (UniqueName: \"kubernetes.io/projected/70965178-adfb-4487-9673-9b6ffe68ad8a-kube-api-access-cx72k\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:49:14.455862 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.455831    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"]
Sep 09 20:49:14.462691 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.462652    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3d3c9036-6d7df7cf49-mjw27"]
Sep 09 20:49:14.473827 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.473803    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"]
Sep 09 20:49:14.479879 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.479860    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3d3c9036-77c97b5774-lh985"]
Sep 09 20:49:14.751500 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.751466    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:14.854458 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.854424    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:14.950831 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:14.950799    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:15.732434 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:15.732397    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:16.258239 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:16.258203    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:16.280071 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:16.280036    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7" path="/var/lib/kubelet/pods/0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7/volumes"
Sep 09 20:49:16.280495 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:16.280476    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70965178-adfb-4487-9673-9b6ffe68ad8a" path="/var/lib/kubelet/pods/70965178-adfb-4487-9673-9b6ffe68ad8a/volumes"
Sep 09 20:49:22.016551 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:22.016518    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:22.122808 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:22.122765    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:22.247472 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:22.247434    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:22.346040 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:22.345924    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:22.692215 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:22.692110    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:25.199435 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.199394    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"]
Sep 09 20:49:25.199860 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.199838    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70965178-adfb-4487-9673-9b6ffe68ad8a" containerName="payload-processing"
Sep 09 20:49:25.199860 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.199855    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="70965178-adfb-4487-9673-9b6ffe68ad8a" containerName="payload-processing"
Sep 09 20:49:25.200044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.199878    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7" containerName="payload-pre-processing"
Sep 09 20:49:25.200044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.199884    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7" containerName="payload-pre-processing"
Sep 09 20:49:25.200044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.199949    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cd5820b-8e4c-4153-bc9b-cf5fafa22ef7" containerName="payload-pre-processing"
Sep 09 20:49:25.200044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.199962    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="70965178-adfb-4487-9673-9b6ffe68ad8a" containerName="payload-processing"
Sep 09 20:49:25.204229 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.204211    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.207256 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.207237    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c7643f\"/\"openshift-service-ca.crt\""
Sep 09 20:49:25.208120 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.208087    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c7643f\"/\"default-dockercfg-f9ssc\""
Sep 09 20:49:25.208435 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.208418    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c7643f\"/\"test-model-c7643f-kserve-self-signed-certs\""
Sep 09 20:49:25.208502 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.208418    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c7643f\"/\"kube-root-ca.crt\""
Sep 09 20:49:25.223110 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.223085    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"]
Sep 09 20:49:25.228397 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.228362    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:25.273287 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.273265    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-home\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.273405 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.273318    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22eeea6-2581-4705-a6db-1e17b960fc8b-tls-certs\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.273405 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.273353    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-model-cache\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.273405 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.273378    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-tmp-dir\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.273592 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.273408    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdlg4\" (UniqueName: \"kubernetes.io/projected/c22eeea6-2581-4705-a6db-1e17b960fc8b-kube-api-access-sdlg4\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.273592 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.273448    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-dshm\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.374158 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.374123    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-dshm\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.374364 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.374233    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-home\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.374364 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.374264    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22eeea6-2581-4705-a6db-1e17b960fc8b-tls-certs\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.374364 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.374308    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-model-cache\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.374364 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.374335    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-tmp-dir\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.374548 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.374373    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdlg4\" (UniqueName: \"kubernetes.io/projected/c22eeea6-2581-4705-a6db-1e17b960fc8b-kube-api-access-sdlg4\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.374680 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.374651    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-home\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.374822 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.374686    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-model-cache\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.374822 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.374723    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-tmp-dir\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.376544 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.376521    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-dshm\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.376673 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.376586    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22eeea6-2581-4705-a6db-1e17b960fc8b-tls-certs\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.381858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.381839    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdlg4\" (UniqueName: \"kubernetes.io/projected/c22eeea6-2581-4705-a6db-1e17b960fc8b-kube-api-access-sdlg4\") pod \"test-model-c7643f-kserve-c5fdf9459-jvjsg\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") " pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.513187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.513136    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:25.784381 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:25.784350    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"]
Sep 09 20:49:25.785797 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:49:25.785770    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc22eeea6_2581_4705_a6db_1e17b960fc8b.slice/crio-38e9f85e40a1de0b4b9d6df2f48ff95ff6d486c494980478be9efeae8a9ffa76 WatchSource:0}: Error finding container 38e9f85e40a1de0b4b9d6df2f48ff95ff6d486c494980478be9efeae8a9ffa76: Status 404 returned error can't find the container with id 38e9f85e40a1de0b4b9d6df2f48ff95ff6d486c494980478be9efeae8a9ffa76
Sep 09 20:49:26.207038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:26.206950    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg" event={"ID":"c22eeea6-2581-4705-a6db-1e17b960fc8b","Type":"ContainerStarted","Data":"47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912"}
Sep 09 20:49:26.207038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:26.206996    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg" event={"ID":"c22eeea6-2581-4705-a6db-1e17b960fc8b","Type":"ContainerStarted","Data":"38e9f85e40a1de0b4b9d6df2f48ff95ff6d486c494980478be9efeae8a9ffa76"}
Sep 09 20:49:26.225358 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:26.225305    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg" podStartSLOduration=1.225289275 podStartE2EDuration="1.225289275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:49:26.223988205 +0000 UTC m=+1026.374660548" watchObservedRunningTime="2026-09-09 20:49:26.225289275 +0000 UTC m=+1026.375961637"
Sep 09 20:49:26.947064 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:26.947031    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:35.514217 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:35.514155    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:35.514660 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:35.514255    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:35.527123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:35.527096    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:36.279523 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:36.279496    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:49:37.439425 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:37.439393    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:45.925841 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:45.925803    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:47.331635 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:47.331593    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:48.574627 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:48.574595    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:50.938882 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:50.938841    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:52.489541 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:52.489509    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:52.739418 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:52.739373    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:53.145857 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:53.145793    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:53.459801 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:53.459770    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:53.582480 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:53.582446    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:54.750351 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:54.750320    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:54.865677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:54.865641    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:54.959395 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:54.959361    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:56.782254 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:56.782219    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:56.874693 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:56.874660    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:57.966017 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:57.965976    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:58.068282 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.068245    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:58.273065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.272982    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:58.546346 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.546265    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:58.689561 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.689528    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:58.753997 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.753954    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"]
Sep 09 20:49:58.758603 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.758578    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:49:58.761797 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.761774    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-2d6f6c-dockercfg-ppp5m\""
Sep 09 20:49:58.762529 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.762511    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-2d6f6c\""
Sep 09 20:49:58.774370 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.774349    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"]
Sep 09 20:49:58.881392 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.881315    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p929v\" (UniqueName: \"kubernetes.io/projected/09512c78-9da5-412e-99fa-2490e63d9096-kube-api-access-p929v\") pod \"payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr\" (UID: \"09512c78-9da5-412e-99fa-2490e63d9096\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:49:58.881392 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.881362    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09512c78-9da5-412e-99fa-2490e63d9096-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr\" (UID: \"09512c78-9da5-412e-99fa-2490e63d9096\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:49:58.982151 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.982119    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p929v\" (UniqueName: \"kubernetes.io/projected/09512c78-9da5-412e-99fa-2490e63d9096-kube-api-access-p929v\") pod \"payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr\" (UID: \"09512c78-9da5-412e-99fa-2490e63d9096\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:49:58.982563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.982293    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09512c78-9da5-412e-99fa-2490e63d9096-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr\" (UID: \"09512c78-9da5-412e-99fa-2490e63d9096\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:49:58.982967 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.982948    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09512c78-9da5-412e-99fa-2490e63d9096-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr\" (UID: \"09512c78-9da5-412e-99fa-2490e63d9096\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:49:58.990184 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:58.990145    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p929v\" (UniqueName: \"kubernetes.io/projected/09512c78-9da5-412e-99fa-2490e63d9096-kube-api-access-p929v\") pod \"payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr\" (UID: \"09512c78-9da5-412e-99fa-2490e63d9096\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:49:59.069511 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:59.069476    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:49:59.347571 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:59.347540    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"]
Sep 09 20:49:59.348980 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:49:59.348952    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09512c78_9da5_412e_99fa_2490e63d9096.slice/crio-965bc6601902ab76ebf2e33449c936a8e77e6ba8ed7cfd758c43f5d563f0e194 WatchSource:0}: Error finding container 965bc6601902ab76ebf2e33449c936a8e77e6ba8ed7cfd758c43f5d563f0e194: Status 404 returned error can't find the container with id 965bc6601902ab76ebf2e33449c936a8e77e6ba8ed7cfd758c43f5d563f0e194
Sep 09 20:49:59.394932 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:49:59.394905    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr" event={"ID":"09512c78-9da5-412e-99fa-2490e63d9096","Type":"ContainerStarted","Data":"965bc6601902ab76ebf2e33449c936a8e77e6ba8ed7cfd758c43f5d563f0e194"}
Sep 09 20:50:00.402335 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:00.402296    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr" event={"ID":"09512c78-9da5-412e-99fa-2490e63d9096","Type":"ContainerStarted","Data":"6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23"}
Sep 09 20:50:00.402739 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:00.402418    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:50:00.418539 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:00.418487    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr" podStartSLOduration=2.418472726 podStartE2EDuration="2.418472726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:49:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:00.416935273 +0000 UTC m=+1060.567607619" watchObservedRunningTime="2026-09-09 20:50:00.418472726 +0000 UTC m=+1060.569145069"
Sep 09 20:50:10.386889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.386859    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"]
Sep 09 20:50:10.390879 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.390857    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.392892 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.392869    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2b2343\"/\"premium-w0-2b2343-kserve-self-signed-certs\""
Sep 09 20:50:10.401645 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.401625    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"]
Sep 09 20:50:10.419998 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.419968    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:10.493740 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.493712    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-home\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.493867 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.493759    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-tmp-dir\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.493867 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.493781    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjm77\" (UniqueName: \"kubernetes.io/projected/a5e1b010-99bb-42de-94de-618e9cd83e91-kube-api-access-pjm77\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.493867 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.493849    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-model-cache\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.493976 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.493871    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5e1b010-99bb-42de-94de-618e9cd83e91-tls-certs\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.494011 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.493972    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-dshm\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.594755 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.594720    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-model-cache\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.594755 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.594760    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5e1b010-99bb-42de-94de-618e9cd83e91-tls-certs\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.594974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.594820    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-dshm\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.594974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.594843    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-home\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.594974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.594876    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-tmp-dir\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.594974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.594922    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjm77\" (UniqueName: \"kubernetes.io/projected/a5e1b010-99bb-42de-94de-618e9cd83e91-kube-api-access-pjm77\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.595249 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.595226    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-model-cache\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.595347 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.595248    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-home\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.595406 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.595358    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-tmp-dir\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.597072 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.597029    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-dshm\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.597440 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.597421    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5e1b010-99bb-42de-94de-618e9cd83e91-tls-certs\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.602914 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.602896    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjm77\" (UniqueName: \"kubernetes.io/projected/a5e1b010-99bb-42de-94de-618e9cd83e91-kube-api-access-pjm77\") pod \"premium-w0-2b2343-kserve-66497bf486-hkdbn\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") " pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.700728 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.700659    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:10.815343 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.815302    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:10.976730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:10.976709    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"]
Sep 09 20:50:10.977582 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:50:10.977561    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5e1b010_99bb_42de_94de_618e9cd83e91.slice/crio-bfe98055247b88eaab5bdc12ee8a09bbbd9453e9c4dcd8e3ad927af6017c3998 WatchSource:0}: Error finding container bfe98055247b88eaab5bdc12ee8a09bbbd9453e9c4dcd8e3ad927af6017c3998: Status 404 returned error can't find the container with id bfe98055247b88eaab5bdc12ee8a09bbbd9453e9c4dcd8e3ad927af6017c3998
Sep 09 20:50:11.410292 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:11.410223    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:50:11.467938 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:11.467900    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn" event={"ID":"a5e1b010-99bb-42de-94de-618e9cd83e91","Type":"ContainerStarted","Data":"59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0"}
Sep 09 20:50:11.468113 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:11.467950    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn" event={"ID":"a5e1b010-99bb-42de-94de-618e9cd83e91","Type":"ContainerStarted","Data":"bfe98055247b88eaab5bdc12ee8a09bbbd9453e9c4dcd8e3ad927af6017c3998"}
Sep 09 20:50:11.485449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:11.485402    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn" podStartSLOduration=1.485388404 podStartE2EDuration="1.485388404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:50:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:11.484629641 +0000 UTC m=+1071.635301984" watchObservedRunningTime="2026-09-09 20:50:11.485388404 +0000 UTC m=+1071.636060747"
Sep 09 20:50:16.854558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:16.854454    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:16.958618 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:16.958575    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:17.061959 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:17.061924    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:20.701094 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:20.701062    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:20.701479 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:20.701110    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:20.713619 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:20.713595    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:21.536942 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:21.536901    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:50:21.706196 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:21.706152    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:27.590927 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:27.590882    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:28.299071 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:28.299034    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:34.099696 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:34.099665    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:34.173048 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:34.173015    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:34.257424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:34.257382    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:34.357943 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:34.357831    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:35.029107 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:35.029068    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:35.129265 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:35.129230    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:36.637526 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:36.637497    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:42.511455 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:42.511413    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:42.588439 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:42.588405    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:43.172751 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:43.172718    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:49.217202 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:49.217140    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:49.320673 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:49.320643    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:49.423873 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:49.423842    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:49.733069 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:49.733037    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:50.051863 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:50.051777    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:56.035696 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:56.035644    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:56.136415 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:56.136383    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:57.262289 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.262252    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:57.361859 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.361817    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:57.557812 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.557718    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:57.646811 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.646779    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"]
Sep 09 20:50:57.649887 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.649860    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:57.650490 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.650472    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:50:57.654763 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.654743    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-999c8b38\""
Sep 09 20:50:57.654861 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.654743    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-999c8b38-dockercfg-qkjq7\""
Sep 09 20:50:57.661754 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.661735    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"]
Sep 09 20:50:57.685447 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.685421    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"]
Sep 09 20:50:57.689547 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.689525    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:50:57.698309 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.698285    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"]
Sep 09 20:50:57.752685 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.752644    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r4pk\" (UniqueName: \"kubernetes.io/projected/bf315086-cbec-40f5-92d2-e079797c088b-kube-api-access-9r4pk\") pod \"payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7\" (UID: \"bf315086-cbec-40f5-92d2-e079797c088b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:50:57.752966 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.752807    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf315086-cbec-40f5-92d2-e079797c088b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7\" (UID: \"bf315086-cbec-40f5-92d2-e079797c088b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:50:57.854154 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.854064    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf315086-cbec-40f5-92d2-e079797c088b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7\" (UID: \"bf315086-cbec-40f5-92d2-e079797c088b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:50:57.854154 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.854145    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r4pk\" (UniqueName: \"kubernetes.io/projected/bf315086-cbec-40f5-92d2-e079797c088b-kube-api-access-9r4pk\") pod \"payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7\" (UID: \"bf315086-cbec-40f5-92d2-e079797c088b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:50:57.854401 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.854208    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5nf5\" (UniqueName: \"kubernetes.io/projected/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-kube-api-access-q5nf5\") pod \"payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5\" (UID: \"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34\") " pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:50:57.854401 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.854244    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-plugins-config-volume\") pod \"payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5\" (UID: \"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34\") " pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:50:57.855033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.855008    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf315086-cbec-40f5-92d2-e079797c088b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7\" (UID: \"bf315086-cbec-40f5-92d2-e079797c088b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:50:57.862119 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.862090    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r4pk\" (UniqueName: \"kubernetes.io/projected/bf315086-cbec-40f5-92d2-e079797c088b-kube-api-access-9r4pk\") pod \"payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7\" (UID: \"bf315086-cbec-40f5-92d2-e079797c088b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:50:57.955718 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.955675    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5nf5\" (UniqueName: \"kubernetes.io/projected/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-kube-api-access-q5nf5\") pod \"payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5\" (UID: \"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34\") " pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:50:57.955718 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.955720    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-plugins-config-volume\") pod \"payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5\" (UID: \"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34\") " pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:50:57.956498 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.956479    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-plugins-config-volume\") pod \"payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5\" (UID: \"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34\") " pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:50:57.961194 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.961156    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:50:57.963727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:57.963700    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5nf5\" (UniqueName: \"kubernetes.io/projected/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-kube-api-access-q5nf5\") pod \"payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5\" (UID: \"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34\") " pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:50:58.000795 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.000763    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:50:58.022214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.021763    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:58.235293 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.235262    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"]
Sep 09 20:50:58.235735 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:50:58.235711    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf315086_cbec_40f5_92d2_e079797c088b.slice/crio-8894975af264e5fad628aae6caed1736758fc67ec110ef7097ae8ee28ce79ed9 WatchSource:0}: Error finding container 8894975af264e5fad628aae6caed1736758fc67ec110ef7097ae8ee28ce79ed9: Status 404 returned error can't find the container with id 8894975af264e5fad628aae6caed1736758fc67ec110ef7097ae8ee28ce79ed9
Sep 09 20:50:58.287587 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.287559    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"]
Sep 09 20:50:58.288361 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:50:58.288336    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c02ad3_347f_4896_ad3a_e2b6f4fbbe34.slice/crio-ac59792adf837bcb00d65c856a74cb8701f33fb2b723e74e9bb509139c5210be WatchSource:0}: Error finding container ac59792adf837bcb00d65c856a74cb8701f33fb2b723e74e9bb509139c5210be: Status 404 returned error can't find the container with id ac59792adf837bcb00d65c856a74cb8701f33fb2b723e74e9bb509139c5210be
Sep 09 20:50:58.748404 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.748269    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7" event={"ID":"bf315086-cbec-40f5-92d2-e079797c088b","Type":"ContainerStarted","Data":"43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc"}
Sep 09 20:50:58.748404 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.748348    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:50:58.748404 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.748374    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7" event={"ID":"bf315086-cbec-40f5-92d2-e079797c088b","Type":"ContainerStarted","Data":"8894975af264e5fad628aae6caed1736758fc67ec110ef7097ae8ee28ce79ed9"}
Sep 09 20:50:58.750286 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.750259    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5" event={"ID":"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34","Type":"ContainerStarted","Data":"765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9"}
Sep 09 20:50:58.750409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.750308    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5" event={"ID":"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34","Type":"ContainerStarted","Data":"ac59792adf837bcb00d65c856a74cb8701f33fb2b723e74e9bb509139c5210be"}
Sep 09 20:50:58.750409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.750335    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:50:58.768437 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.768387    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7" podStartSLOduration=1.768370215 podStartE2EDuration="1.768370215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:50:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:58.765050833 +0000 UTC m=+1118.915723176" watchObservedRunningTime="2026-09-09 20:50:58.768370215 +0000 UTC m=+1118.919042560"
Sep 09 20:50:58.782852 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:50:58.782811    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5" podStartSLOduration=1.78279828 podStartE2EDuration="1.78279828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:50:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:58.780867238 +0000 UTC m=+1118.931539582" watchObservedRunningTime="2026-09-09 20:50:58.78279828 +0000 UTC m=+1118.933470642"
Sep 09 20:51:09.758459 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:09.758427    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:51:09.758862 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:09.758508    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:51:14.921278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:14.921215    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:20.756405 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:20.756372    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:21.380793 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:21.380755    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:21.504055 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:21.504023    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:21.807322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:21.807289    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:27.642479 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:27.642441    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:27.743564 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:27.743534    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:38.483104 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.483061    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:38.519648 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.519618    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"]
Sep 09 20:51:38.519964 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.519937    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5" podUID="27c02ad3-347f-4896-ad3a-e2b6f4fbbe34" containerName="payload-processing" containerID="cri-o://765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9" gracePeriod=30
Sep 09 20:51:38.527554 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.527526    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"]
Sep 09 20:51:38.528085 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.527755    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7" podUID="bf315086-cbec-40f5-92d2-e079797c088b" containerName="payload-pre-processing" containerID="cri-o://43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc" gracePeriod=30
Sep 09 20:51:38.566953 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.566918    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:38.640870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.640828    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:38.715933 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.715897    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:38.805843 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.805816    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:51:38.808494 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.808475    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:51:38.968092 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.968064    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r4pk\" (UniqueName: \"kubernetes.io/projected/bf315086-cbec-40f5-92d2-e079797c088b-kube-api-access-9r4pk\") pod \"bf315086-cbec-40f5-92d2-e079797c088b\" (UID: \"bf315086-cbec-40f5-92d2-e079797c088b\") "
Sep 09 20:51:38.968273 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.968112    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-plugins-config-volume\") pod \"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34\" (UID: \"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34\") "
Sep 09 20:51:38.968273 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.968140    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5nf5\" (UniqueName: \"kubernetes.io/projected/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-kube-api-access-q5nf5\") pod \"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34\" (UID: \"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34\") "
Sep 09 20:51:38.968273 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.968238    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf315086-cbec-40f5-92d2-e079797c088b-plugins-config-volume\") pod \"bf315086-cbec-40f5-92d2-e079797c088b\" (UID: \"bf315086-cbec-40f5-92d2-e079797c088b\") "
Sep 09 20:51:38.968542 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.968506    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "27c02ad3-347f-4896-ad3a-e2b6f4fbbe34" (UID: "27c02ad3-347f-4896-ad3a-e2b6f4fbbe34"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:51:38.968655 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.968602    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf315086-cbec-40f5-92d2-e079797c088b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf315086-cbec-40f5-92d2-e079797c088b" (UID: "bf315086-cbec-40f5-92d2-e079797c088b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:51:38.969996 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.969977    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf315086-cbec-40f5-92d2-e079797c088b-kube-api-access-9r4pk" (OuterVolumeSpecName: "kube-api-access-9r4pk") pod "bf315086-cbec-40f5-92d2-e079797c088b" (UID: "bf315086-cbec-40f5-92d2-e079797c088b"). InnerVolumeSpecName "kube-api-access-9r4pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:51:38.970109 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.970090    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-kube-api-access-q5nf5" (OuterVolumeSpecName: "kube-api-access-q5nf5") pod "27c02ad3-347f-4896-ad3a-e2b6f4fbbe34" (UID: "27c02ad3-347f-4896-ad3a-e2b6f4fbbe34"). InnerVolumeSpecName "kube-api-access-q5nf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:51:38.983378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.983354    2550 generic.go:356] "Generic (PLEG): container finished" podID="bf315086-cbec-40f5-92d2-e079797c088b" containerID="43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc" exitCode=0
Sep 09 20:51:38.983499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.983399    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"
Sep 09 20:51:38.983499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.983440    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7" event={"ID":"bf315086-cbec-40f5-92d2-e079797c088b","Type":"ContainerDied","Data":"43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc"}
Sep 09 20:51:38.983499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.983490    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7" event={"ID":"bf315086-cbec-40f5-92d2-e079797c088b","Type":"ContainerDied","Data":"8894975af264e5fad628aae6caed1736758fc67ec110ef7097ae8ee28ce79ed9"}
Sep 09 20:51:38.983647 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.983512    2550 scope.go:117] "RemoveContainer" containerID="43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc"
Sep 09 20:51:38.985219 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.984814    2550 generic.go:356] "Generic (PLEG): container finished" podID="27c02ad3-347f-4896-ad3a-e2b6f4fbbe34" containerID="765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9" exitCode=0
Sep 09 20:51:38.985219 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.984846    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5" event={"ID":"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34","Type":"ContainerDied","Data":"765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9"}
Sep 09 20:51:38.985219 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.984862    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"
Sep 09 20:51:38.985219 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.984903    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5" event={"ID":"27c02ad3-347f-4896-ad3a-e2b6f4fbbe34","Type":"ContainerDied","Data":"ac59792adf837bcb00d65c856a74cb8701f33fb2b723e74e9bb509139c5210be"}
Sep 09 20:51:38.993717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.993694    2550 scope.go:117] "RemoveContainer" containerID="43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc"
Sep 09 20:51:38.993980 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:51:38.993961    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc\": container with ID starting with 43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc not found: ID does not exist" containerID="43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc"
Sep 09 20:51:38.994044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.993987    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc"} err="failed to get container status \"43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc\": rpc error: code = NotFound desc = could not find container \"43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc\": container with ID starting with 43f32347db7f7a8e4084efc288c536890404eb12d2ae22fe858187328f1358cc not found: ID does not exist"
Sep 09 20:51:38.994044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:38.994002    2550 scope.go:117] "RemoveContainer" containerID="765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9"
Sep 09 20:51:39.006717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.004332    2550 scope.go:117] "RemoveContainer" containerID="765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9"
Sep 09 20:51:39.007108 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:51:39.007081    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9\": container with ID starting with 765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9 not found: ID does not exist" containerID="765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9"
Sep 09 20:51:39.007251 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.007112    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9"} err="failed to get container status \"765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9\": rpc error: code = NotFound desc = could not find container \"765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9\": container with ID starting with 765a79f86153f2fcdf63947fe865ca635c7ef5c9161d110456f51e8dec4c8ab9 not found: ID does not exist"
Sep 09 20:51:39.011456 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.011434    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"]
Sep 09 20:51:39.016202 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.016175    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-999c8b38-7d558b8bb-65bx7"]
Sep 09 20:51:39.029840 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.029789    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"]
Sep 09 20:51:39.038026 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.038008    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-999c8b38-5675f9b97c-x8dt5"]
Sep 09 20:51:39.064831 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.064804    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:39.068946 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.068926    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf315086-cbec-40f5-92d2-e079797c088b-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:51:39.069020 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.068950    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r4pk\" (UniqueName: \"kubernetes.io/projected/bf315086-cbec-40f5-92d2-e079797c088b-kube-api-access-9r4pk\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:51:39.069020 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.068962    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:51:39.069020 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:39.068971    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5nf5\" (UniqueName: \"kubernetes.io/projected/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34-kube-api-access-q5nf5\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:51:40.279601 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:40.279570    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27c02ad3-347f-4896-ad3a-e2b6f4fbbe34" path="/var/lib/kubelet/pods/27c02ad3-347f-4896-ad3a-e2b6f4fbbe34/volumes"
Sep 09 20:51:40.279953 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:40.279922    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf315086-cbec-40f5-92d2-e079797c088b" path="/var/lib/kubelet/pods/bf315086-cbec-40f5-92d2-e079797c088b/volumes"
Sep 09 20:51:41.478379 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.478340    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"]
Sep 09 20:51:41.478932 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.478912    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c02ad3-347f-4896-ad3a-e2b6f4fbbe34" containerName="payload-processing"
Sep 09 20:51:41.479019 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.478936    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c02ad3-347f-4896-ad3a-e2b6f4fbbe34" containerName="payload-processing"
Sep 09 20:51:41.479019 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.478965    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf315086-cbec-40f5-92d2-e079797c088b" containerName="payload-pre-processing"
Sep 09 20:51:41.479019 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.478976    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf315086-cbec-40f5-92d2-e079797c088b" containerName="payload-pre-processing"
Sep 09 20:51:41.479198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.479088    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf315086-cbec-40f5-92d2-e079797c088b" containerName="payload-pre-processing"
Sep 09 20:51:41.479198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.479105    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c02ad3-347f-4896-ad3a-e2b6f4fbbe34" containerName="payload-processing"
Sep 09 20:51:41.483601 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.483581    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.485658 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.485640    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-2d6f6c\"/\"kube-root-ca.crt\""
Sep 09 20:51:41.485757 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.485641    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-2d6f6c\"/\"openshift-service-ca.crt\""
Sep 09 20:51:41.486217 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.486201    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-2d6f6c\"/\"default-dockercfg-rgh22\""
Sep 09 20:51:41.486276 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.486263    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-2d6f6c\"/\"explicit-model-2d6f6c-kserve-self-signed-certs\""
Sep 09 20:51:41.489960 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.489935    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"]
Sep 09 20:51:41.591603 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.591568    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tls-certs\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.591785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.591610    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-model-cache\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.591785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.591639    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tmp-dir\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.591785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.591660    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-dshm\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.591785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.591689    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-home\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.591785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.591724    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxcrm\" (UniqueName: \"kubernetes.io/projected/a51d0ad1-5348-4cc2-a872-13f7a25f576c-kube-api-access-wxcrm\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.692613 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.692580    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tls-certs\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.692814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.692620    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-model-cache\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.692814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.692658    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tmp-dir\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.692814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.692685    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-dshm\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.692814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.692729    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-home\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.692814 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.692790    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxcrm\" (UniqueName: \"kubernetes.io/projected/a51d0ad1-5348-4cc2-a872-13f7a25f576c-kube-api-access-wxcrm\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.693130 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.693098    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-model-cache\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.693227 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.693203    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tmp-dir\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.693276 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.693259    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-home\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.694974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.694955    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-dshm\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.695130 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.695113    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tls-certs\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.700584 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.700562    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxcrm\" (UniqueName: \"kubernetes.io/projected/a51d0ad1-5348-4cc2-a872-13f7a25f576c-kube-api-access-wxcrm\") pod \"explicit-model-2d6f6c-kserve-77545f559f-f2wx8\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") " pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:41.787676 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.787589    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:41.793494 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:41.793473    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:42.070844 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:42.070819    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"]
Sep 09 20:51:42.072410 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:51:42.072375    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda51d0ad1_5348_4cc2_a872_13f7a25f576c.slice/crio-92438bbac45de9d943029c8f79b87827070fd4868a364fa8c367d30874a0645e WatchSource:0}: Error finding container 92438bbac45de9d943029c8f79b87827070fd4868a364fa8c367d30874a0645e: Status 404 returned error can't find the container with id 92438bbac45de9d943029c8f79b87827070fd4868a364fa8c367d30874a0645e
Sep 09 20:51:43.018902 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:43.018823    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8" event={"ID":"a51d0ad1-5348-4cc2-a872-13f7a25f576c","Type":"ContainerStarted","Data":"5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1"}
Sep 09 20:51:43.018902 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:43.018863    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8" event={"ID":"a51d0ad1-5348-4cc2-a872-13f7a25f576c","Type":"ContainerStarted","Data":"92438bbac45de9d943029c8f79b87827070fd4868a364fa8c367d30874a0645e"}
Sep 09 20:51:43.036194 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:43.036129    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8" podStartSLOduration=2.036110849 podStartE2EDuration="2.036110849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:51:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:43.033383117 +0000 UTC m=+1163.184055459" watchObservedRunningTime="2026-09-09 20:51:43.036110849 +0000 UTC m=+1163.186783192"
Sep 09 20:51:44.913704 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:44.913668    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:45.018692 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:45.018661    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:45.161442 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:45.161409    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:45.573535 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:45.573494    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:45.671654 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:45.671620    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:45.766214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:45.766181    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:46.621574 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:46.621546    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:51.381219 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:51.381186    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:51.728808 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:51.728774    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:51.793903 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:51.793877    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:51.793903 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:51.793910    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:51.807277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:51.807254    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:51.836351 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:51.836318    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:51.929314 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:51.929278    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:52.084910 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:52.084825    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:51:52.090208 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:52.090159    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:57.697624 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:57.697582    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:57.798630 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:57.798597    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:58.654782 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:58.654742    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:58.979117 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:58.979081    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"]
Sep 09 20:51:58.983334 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:58.983313    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:58.985568 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:58.985544    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-78a7d5-openshift-default-dockercfg-sgtl9\""
Sep 09 20:51:58.999160 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:58.999134    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"]
Sep 09 20:51:59.023669 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.023637    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:59.053974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.053943    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh8zx\" (UniqueName: \"kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-kube-api-access-kh8zx\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.054198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.053999    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-socket\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.054198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.054078    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istiod-ca-cert\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.054198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.054127    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-envoy\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.054403 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.054250    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-podinfo\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.054403 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.054282    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-data\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.054403 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.054357    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-certs\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.054403 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.054380    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-token\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.054554 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.054433    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-credential-socket\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155057 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155025    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-podinfo\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155057 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155063    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-data\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155098    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-certs\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155114    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-token\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155144    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-credential-socket\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155187    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh8zx\" (UniqueName: \"kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-kube-api-access-kh8zx\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155234    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-socket\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155271    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istiod-ca-cert\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155302    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-envoy\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155705 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155567    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-certs\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155705 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155645    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-credential-socket\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155705 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155662    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-data\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.155982 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.155961    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-socket\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.156061 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.156045    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istiod-ca-cert\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.157668 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.157653    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-envoy\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.157735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.157688    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-podinfo\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.164539 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.164514    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-token\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.164743 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.164723    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh8zx\" (UniqueName: \"kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-kube-api-access-kh8zx\") pod \"e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") " pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.293990 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.293901    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:51:59.301524 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.301494    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:59.601569 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.601542    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"]
Sep 09 20:51:59.603762 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:51:59.603734    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a332682_7abd_4c7a_8fad_0e3af5df2b7b.slice/crio-0a183d152c9b8bbe564531c5d1dda207efa913296e5d5db01e2f3bac70a6ea80 WatchSource:0}: Error finding container 0a183d152c9b8bbe564531c5d1dda207efa913296e5d5db01e2f3bac70a6ea80: Status 404 returned error can't find the container with id 0a183d152c9b8bbe564531c5d1dda207efa913296e5d5db01e2f3bac70a6ea80
Sep 09 20:51:59.606247 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.606195    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:51:59.606337 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.606293    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:51:59.606376 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:51:59.606337    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:52:00.120578 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:00.120540    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" event={"ID":"4a332682-7abd-4c7a-8fad-0e3af5df2b7b","Type":"ContainerStarted","Data":"0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0"}
Sep 09 20:52:00.120974 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:00.120625    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" event={"ID":"4a332682-7abd-4c7a-8fad-0e3af5df2b7b","Type":"ContainerStarted","Data":"0a183d152c9b8bbe564531c5d1dda207efa913296e5d5db01e2f3bac70a6ea80"}
Sep 09 20:52:00.144395 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:00.144347    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" podStartSLOduration=2.144331157 podStartE2EDuration="2.144331157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:00.142504931 +0000 UTC m=+1180.293177274" watchObservedRunningTime="2026-09-09 20:52:00.144331157 +0000 UTC m=+1180.295003499"
Sep 09 20:52:00.294417 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:00.294387    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:52:00.299938 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:00.299915    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:52:01.127049 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:01.127017    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:52:01.127916 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:01.127898    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:52:02.077514 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:02.077480    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:02.163649 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:02.163616    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:03.354081 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:03.354047    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:03.452540 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:03.452505    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:03.840226 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:03.840194    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:03.943251 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:03.943213    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:04.043407 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.043372    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"]
Sep 09 20:52:04.046951 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.046935    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 20:52:04.049423 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.049404    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-78a7d5-dockercfg-6x27p\""
Sep 09 20:52:04.049517 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.049490    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-78a7d5\""
Sep 09 20:52:04.058053 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.058028    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"]
Sep 09 20:52:04.107895 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.106406    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85qqw\" (UniqueName: \"kubernetes.io/projected/84634948-7d85-413e-bd88-238885c7cf16-kube-api-access-85qqw\") pod \"payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87\" (UID: \"84634948-7d85-413e-bd88-238885c7cf16\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 20:52:04.107895 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.106516    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84634948-7d85-413e-bd88-238885c7cf16-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87\" (UID: \"84634948-7d85-413e-bd88-238885c7cf16\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 20:52:04.208294 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.208258    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84634948-7d85-413e-bd88-238885c7cf16-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87\" (UID: \"84634948-7d85-413e-bd88-238885c7cf16\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 20:52:04.208485 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.208393    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85qqw\" (UniqueName: \"kubernetes.io/projected/84634948-7d85-413e-bd88-238885c7cf16-kube-api-access-85qqw\") pod \"payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87\" (UID: \"84634948-7d85-413e-bd88-238885c7cf16\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 20:52:04.209000 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.208975    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84634948-7d85-413e-bd88-238885c7cf16-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87\" (UID: \"84634948-7d85-413e-bd88-238885c7cf16\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 20:52:04.217491 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.217466    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85qqw\" (UniqueName: \"kubernetes.io/projected/84634948-7d85-413e-bd88-238885c7cf16-kube-api-access-85qqw\") pod \"payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87\" (UID: \"84634948-7d85-413e-bd88-238885c7cf16\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 20:52:04.358781 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.358707    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 20:52:04.634346 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:04.634311    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"]
Sep 09 20:52:04.635291 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:52:04.635258    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84634948_7d85_413e_bd88_238885c7cf16.slice/crio-351c63388a9d2c68c68d43bebc62a1fc00f1f4240bcef9ae15316d7a6838a13a WatchSource:0}: Error finding container 351c63388a9d2c68c68d43bebc62a1fc00f1f4240bcef9ae15316d7a6838a13a: Status 404 returned error can't find the container with id 351c63388a9d2c68c68d43bebc62a1fc00f1f4240bcef9ae15316d7a6838a13a
Sep 09 20:52:05.155940 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:05.155900    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87" event={"ID":"84634948-7d85-413e-bd88-238885c7cf16","Type":"ContainerStarted","Data":"3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9"}
Sep 09 20:52:05.156111 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:05.156003    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 20:52:05.156111 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:05.156025    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87" event={"ID":"84634948-7d85-413e-bd88-238885c7cf16","Type":"ContainerStarted","Data":"351c63388a9d2c68c68d43bebc62a1fc00f1f4240bcef9ae15316d7a6838a13a"}
Sep 09 20:52:05.174034 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:05.173979    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87" podStartSLOduration=1.173963733 podStartE2EDuration="1.173963733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:05.170772543 +0000 UTC m=+1185.321444887" watchObservedRunningTime="2026-09-09 20:52:05.173963733 +0000 UTC m=+1185.324636137"
Sep 09 20:52:08.256025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:08.255986    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:10.039604 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:10.039569    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:13.493635 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:13.493597    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:14.711195 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:14.711144    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:16.164592 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:16.164561    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 20:52:20.351061 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:20.351020    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:52:20.353074 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:20.353052    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:52:25.124082 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.124033    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:25.171074 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.171047    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"]
Sep 09 20:52:25.171388 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.171352    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr" podUID="09512c78-9da5-412e-99fa-2490e63d9096" containerName="payload-pre-processing" containerID="cri-o://6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23" gracePeriod=30
Sep 09 20:52:25.205086 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.202381    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:25.283125 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.281553    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:25.386274 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.386147    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:25.457265 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.457236    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:25.523718 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.523642    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:25.552086 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.552060    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:52:25.622717 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.622685    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p929v\" (UniqueName: \"kubernetes.io/projected/09512c78-9da5-412e-99fa-2490e63d9096-kube-api-access-p929v\") pod \"09512c78-9da5-412e-99fa-2490e63d9096\" (UID: \"09512c78-9da5-412e-99fa-2490e63d9096\") "
Sep 09 20:52:25.622889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.622804    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09512c78-9da5-412e-99fa-2490e63d9096-plugins-config-volume\") pod \"09512c78-9da5-412e-99fa-2490e63d9096\" (UID: \"09512c78-9da5-412e-99fa-2490e63d9096\") "
Sep 09 20:52:25.623189 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.623136    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09512c78-9da5-412e-99fa-2490e63d9096-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09512c78-9da5-412e-99fa-2490e63d9096" (UID: "09512c78-9da5-412e-99fa-2490e63d9096"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:52:25.624755 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.624729    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09512c78-9da5-412e-99fa-2490e63d9096-kube-api-access-p929v" (OuterVolumeSpecName: "kube-api-access-p929v") pod "09512c78-9da5-412e-99fa-2490e63d9096" (UID: "09512c78-9da5-412e-99fa-2490e63d9096"). InnerVolumeSpecName "kube-api-access-p929v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:52:25.723811 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.723782    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09512c78-9da5-412e-99fa-2490e63d9096-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:52:25.723811 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.723806    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p929v\" (UniqueName: \"kubernetes.io/projected/09512c78-9da5-412e-99fa-2490e63d9096-kube-api-access-p929v\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:52:25.873609 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:25.873569    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:26.288421 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:26.288382    2550 generic.go:356] "Generic (PLEG): container finished" podID="09512c78-9da5-412e-99fa-2490e63d9096" containerID="6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23" exitCode=0
Sep 09 20:52:26.288421 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:26.288420    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"
Sep 09 20:52:26.288949 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:26.288446    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr" event={"ID":"09512c78-9da5-412e-99fa-2490e63d9096","Type":"ContainerDied","Data":"6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23"}
Sep 09 20:52:26.288949 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:26.288505    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr" event={"ID":"09512c78-9da5-412e-99fa-2490e63d9096","Type":"ContainerDied","Data":"965bc6601902ab76ebf2e33449c936a8e77e6ba8ed7cfd758c43f5d563f0e194"}
Sep 09 20:52:26.288949 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:26.288537    2550 scope.go:117] "RemoveContainer" containerID="6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23"
Sep 09 20:52:26.299615 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:26.299595    2550 scope.go:117] "RemoveContainer" containerID="6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23"
Sep 09 20:52:26.299945 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:52:26.299922    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23\": container with ID starting with 6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23 not found: ID does not exist" containerID="6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23"
Sep 09 20:52:26.300038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:26.299951    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23"} err="failed to get container status \"6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23\": rpc error: code = NotFound desc = could not find container \"6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23\": container with ID starting with 6f10f4e680a0eb3df388fefdb367738294ae3bbab80a0a8e0d5e7ec88074ea23 not found: ID does not exist"
Sep 09 20:52:26.307894 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:26.307872    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"]
Sep 09 20:52:26.312065 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:26.312043    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d6f6c-575dbb4f5d-p4knr"]
Sep 09 20:52:26.958975 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:26.958937    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:27.257692 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:27.257600    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:27.562262 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:27.560708    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:27.696485 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:27.696448    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:28.280346 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:28.280315    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09512c78-9da5-412e-99fa-2490e63d9096" path="/var/lib/kubelet/pods/09512c78-9da5-412e-99fa-2490e63d9096/volumes"
Sep 09 20:52:30.677859 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:30.677811    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:30.775695 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:30.775658    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"]
Sep 09 20:52:30.808334 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:30.808300    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:30.876839 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:30.876804    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:31.854184 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:31.848732    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:31.946428 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:31.946387    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:32.052043 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.052010    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:32.357340 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.357307    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:32.458462 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.458426    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:32.503938 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.503859    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"]
Sep 09 20:52:32.504370 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.504352    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09512c78-9da5-412e-99fa-2490e63d9096" containerName="payload-pre-processing"
Sep 09 20:52:32.504370 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.504371    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="09512c78-9da5-412e-99fa-2490e63d9096" containerName="payload-pre-processing"
Sep 09 20:52:32.504523 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.504494    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="09512c78-9da5-412e-99fa-2490e63d9096" containerName="payload-pre-processing"
Sep 09 20:52:32.507557 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.507538    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 20:52:32.510001 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.509978    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-3fa05f\""
Sep 09 20:52:32.510112 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.509988    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-3fa05f-dockercfg-c9tb4\""
Sep 09 20:52:32.519825 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.519804    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"]
Sep 09 20:52:32.611108 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.611068    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3954a225-0a11-41a0-8ee6-321448e62e10-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq\" (UID: \"3954a225-0a11-41a0-8ee6-321448e62e10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 20:52:32.611416 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.611390    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrh9f\" (UniqueName: \"kubernetes.io/projected/3954a225-0a11-41a0-8ee6-321448e62e10-kube-api-access-zrh9f\") pod \"payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq\" (UID: \"3954a225-0a11-41a0-8ee6-321448e62e10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 20:52:32.712839 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.712809    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3954a225-0a11-41a0-8ee6-321448e62e10-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq\" (UID: \"3954a225-0a11-41a0-8ee6-321448e62e10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 20:52:32.713058 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.712855    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrh9f\" (UniqueName: \"kubernetes.io/projected/3954a225-0a11-41a0-8ee6-321448e62e10-kube-api-access-zrh9f\") pod \"payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq\" (UID: \"3954a225-0a11-41a0-8ee6-321448e62e10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 20:52:32.713499 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.713480    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3954a225-0a11-41a0-8ee6-321448e62e10-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq\" (UID: \"3954a225-0a11-41a0-8ee6-321448e62e10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 20:52:32.721910 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.721881    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrh9f\" (UniqueName: \"kubernetes.io/projected/3954a225-0a11-41a0-8ee6-321448e62e10-kube-api-access-zrh9f\") pod \"payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq\" (UID: \"3954a225-0a11-41a0-8ee6-321448e62e10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 20:52:32.819470 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:32.819390    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 20:52:33.106873 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:33.106839    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"]
Sep 09 20:52:33.107310 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:52:33.107229    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3954a225_0a11_41a0_8ee6_321448e62e10.slice/crio-ae7f10693e1f5ffc3105958ef9471fdaad0946c45a7e5fcc2338ca98e5730acd WatchSource:0}: Error finding container ae7f10693e1f5ffc3105958ef9471fdaad0946c45a7e5fcc2338ca98e5730acd: Status 404 returned error can't find the container with id ae7f10693e1f5ffc3105958ef9471fdaad0946c45a7e5fcc2338ca98e5730acd
Sep 09 20:52:33.335924 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:33.335883    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq" event={"ID":"3954a225-0a11-41a0-8ee6-321448e62e10","Type":"ContainerStarted","Data":"6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c"}
Sep 09 20:52:33.336102 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:33.335951    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq" event={"ID":"3954a225-0a11-41a0-8ee6-321448e62e10","Type":"ContainerStarted","Data":"ae7f10693e1f5ffc3105958ef9471fdaad0946c45a7e5fcc2338ca98e5730acd"}
Sep 09 20:52:33.336471 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:33.336272    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 20:52:33.356366 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:33.356260    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq" podStartSLOduration=1.356241115 podStartE2EDuration="1.356241115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:33.351797269 +0000 UTC m=+1213.502469613" watchObservedRunningTime="2026-09-09 20:52:33.356241115 +0000 UTC m=+1213.506913458"
Sep 09 20:52:36.357341 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:36.357309    2550 generic.go:356] "Generic (PLEG): container finished" podID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerID="0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0" exitCode=0
Sep 09 20:52:36.357863 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:36.357384    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" event={"ID":"4a332682-7abd-4c7a-8fad-0e3af5df2b7b","Type":"ContainerDied","Data":"0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0"}
Sep 09 20:52:36.357934 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:36.357897    2550 scope.go:117] "RemoveContainer" containerID="0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0"
Sep 09 20:52:36.359030 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:36.359000    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:52:36.359151 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:36.359086    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:52:36.359151 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:36.359133    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:52:37.366528 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:37.366495    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" event={"ID":"4a332682-7abd-4c7a-8fad-0e3af5df2b7b","Type":"ContainerStarted","Data":"e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc"}
Sep 09 20:52:38.294708 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:38.294676    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:52:38.294999 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:38.294976    2550 patch_prober.go:28] interesting pod/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 20:52:38.295049 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:38.295029    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 20:52:39.294385 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:39.294350    2550 patch_prober.go:28] interesting pod/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 20:52:39.294755 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:39.294404    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 20:52:40.294847 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:40.294812    2550 patch_prober.go:28] interesting pod/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 20:52:40.295260 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:40.294873    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 20:52:41.295002 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:41.294962    2550 patch_prober.go:28] interesting pod/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 20:52:41.295504 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:41.295023    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 20:52:42.294406 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:42.294371    2550 patch_prober.go:28] interesting pod/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 20:52:42.294610 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:42.294447    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 20:52:43.300016 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:43.299981    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:52:43.300443 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:43.300346    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:52:43.301228 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:43.301207    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 20:52:43.392490 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:43.392453    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:44.345804 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:44.345775    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 20:52:44.982449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:44.982415    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:45.796006 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:45.795969    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8" podUID="a51d0ad1-5348-4cc2-a872-13f7a25f576c" containerName="main" containerID="cri-o://5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1" gracePeriod=45
Sep 09 20:52:46.052214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.052128    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:52:46.159729 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.159702    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tmp-dir\") pod \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") "
Sep 09 20:52:46.159908 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.159737    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxcrm\" (UniqueName: \"kubernetes.io/projected/a51d0ad1-5348-4cc2-a872-13f7a25f576c-kube-api-access-wxcrm\") pod \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") "
Sep 09 20:52:46.159908 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.159783    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tls-certs\") pod \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") "
Sep 09 20:52:46.159908 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.159856    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-dshm\") pod \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") "
Sep 09 20:52:46.160077 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.159930    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a51d0ad1-5348-4cc2-a872-13f7a25f576c" (UID: "a51d0ad1-5348-4cc2-a872-13f7a25f576c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:52:46.160077 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.159950    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-home\") pod \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") "
Sep 09 20:52:46.160077 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.160011    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-model-cache\") pod \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\" (UID: \"a51d0ad1-5348-4cc2-a872-13f7a25f576c\") "
Sep 09 20:52:46.160246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.160206    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-home" (OuterVolumeSpecName: "home") pod "a51d0ad1-5348-4cc2-a872-13f7a25f576c" (UID: "a51d0ad1-5348-4cc2-a872-13f7a25f576c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:52:46.160303 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.160285    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-model-cache" (OuterVolumeSpecName: "model-cache") pod "a51d0ad1-5348-4cc2-a872-13f7a25f576c" (UID: "a51d0ad1-5348-4cc2-a872-13f7a25f576c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:52:46.160398 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.160382    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:52:46.160458 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.160403    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:52:46.160458 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.160418    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:52:46.162051 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.162012    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a51d0ad1-5348-4cc2-a872-13f7a25f576c" (UID: "a51d0ad1-5348-4cc2-a872-13f7a25f576c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:52:46.162051 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.162037    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a51d0ad1-5348-4cc2-a872-13f7a25f576c-kube-api-access-wxcrm" (OuterVolumeSpecName: "kube-api-access-wxcrm") pod "a51d0ad1-5348-4cc2-a872-13f7a25f576c" (UID: "a51d0ad1-5348-4cc2-a872-13f7a25f576c"). InnerVolumeSpecName "kube-api-access-wxcrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:52:46.162284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.162083    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-dshm" (OuterVolumeSpecName: "dshm") pod "a51d0ad1-5348-4cc2-a872-13f7a25f576c" (UID: "a51d0ad1-5348-4cc2-a872-13f7a25f576c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:52:46.261661 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.261630    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51d0ad1-5348-4cc2-a872-13f7a25f576c-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:52:46.261661 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.261656    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxcrm\" (UniqueName: \"kubernetes.io/projected/a51d0ad1-5348-4cc2-a872-13f7a25f576c-kube-api-access-wxcrm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:52:46.261661 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.261668    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51d0ad1-5348-4cc2-a872-13f7a25f576c-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:52:46.423100 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.423017    2550 generic.go:356] "Generic (PLEG): container finished" podID="a51d0ad1-5348-4cc2-a872-13f7a25f576c" containerID="5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1" exitCode=0
Sep 09 20:52:46.423100 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.423059    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"
Sep 09 20:52:46.423375 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.423107    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8" event={"ID":"a51d0ad1-5348-4cc2-a872-13f7a25f576c","Type":"ContainerDied","Data":"5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1"}
Sep 09 20:52:46.423375 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.423155    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8" event={"ID":"a51d0ad1-5348-4cc2-a872-13f7a25f576c","Type":"ContainerDied","Data":"92438bbac45de9d943029c8f79b87827070fd4868a364fa8c367d30874a0645e"}
Sep 09 20:52:46.423375 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.423203    2550 scope.go:117] "RemoveContainer" containerID="5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1"
Sep 09 20:52:46.432244 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.432223    2550 scope.go:117] "RemoveContainer" containerID="5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1"
Sep 09 20:52:46.432581 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:52:46.432561    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1\": container with ID starting with 5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1 not found: ID does not exist" containerID="5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1"
Sep 09 20:52:46.432664 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.432593    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1"} err="failed to get container status \"5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1\": rpc error: code = NotFound desc = could not find container \"5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1\": container with ID starting with 5218836689dca892dad6880e7d0ddd6a84ae4c85cf290e210d95b8119068fab1 not found: ID does not exist"
Sep 09 20:52:46.439533 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.439510    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"]
Sep 09 20:52:46.443276 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:46.443254    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-2d6f6c/explicit-model-2d6f6c-kserve-77545f559f-f2wx8"]
Sep 09 20:52:48.280314 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:48.280285    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a51d0ad1-5348-4cc2-a872-13f7a25f576c" path="/var/lib/kubelet/pods/a51d0ad1-5348-4cc2-a872-13f7a25f576c/volumes"
Sep 09 20:52:49.494135 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:49.494090    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:54.057058 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:54.057025    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:55.542778 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:52:55.542747    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:07.654710 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:07.654675    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:07.891299 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:07.891270    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:08.605343 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.605306    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"]
Sep 09 20:53:08.605727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.605714    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a51d0ad1-5348-4cc2-a872-13f7a25f576c" containerName="main"
Sep 09 20:53:08.605771 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.605728    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a51d0ad1-5348-4cc2-a872-13f7a25f576c" containerName="main"
Sep 09 20:53:08.605848 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.605804    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a51d0ad1-5348-4cc2-a872-13f7a25f576c" containerName="main"
Sep 09 20:53:08.608769 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.608752    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.610925 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.610904    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-a214cb-openshift-default-dockercfg-v7nc6\""
Sep 09 20:53:08.620409 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.620385    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"]
Sep 09 20:53:08.679898 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.679866    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-certs\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.680372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.679908    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-socket\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.680372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.679932    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-podinfo\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.680372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.680041    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fqcm\" (UniqueName: \"kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-kube-api-access-2fqcm\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.680372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.680091    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-envoy\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.680372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.680197    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istiod-ca-cert\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.680372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.680237    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-credential-socket\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.680372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.680274    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-token\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.680372 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.680339    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-data\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781143 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781101    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-podinfo\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781192    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fqcm\" (UniqueName: \"kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-kube-api-access-2fqcm\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781222    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-envoy\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781263    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istiod-ca-cert\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781336    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-credential-socket\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781596 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781386    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-token\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781596 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781416    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-data\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781596 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781490    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-certs\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781596 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781521    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-socket\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781784    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-credential-socket\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781865 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781833    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-socket\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781920 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781893    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-data\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.781978 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.781936    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-certs\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.782278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.782257    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istiod-ca-cert\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.783665 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.783642    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-envoy\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.783762 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.783691    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-podinfo\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.792727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.792702    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-token\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.793110 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.793092    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fqcm\" (UniqueName: \"kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-kube-api-access-2fqcm\") pod \"e2e-noten-a214cb-openshift-default-95b87f746-5x8lt\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") " pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.817358 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.817330    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:08.918911 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.918823    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:08.919074 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:08.918912    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:09.210032 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:09.210010    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"]
Sep 09 20:53:09.211628 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:53:09.211602    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de424bb_588e_4a60_a6a3_5bacf8e71f1e.slice/crio-d66afe15f009b7eae787753baa990cab357e0486f0940d4e97f413669b9b592d WatchSource:0}: Error finding container d66afe15f009b7eae787753baa990cab357e0486f0940d4e97f413669b9b592d: Status 404 returned error can't find the container with id d66afe15f009b7eae787753baa990cab357e0486f0940d4e97f413669b9b592d
Sep 09 20:53:09.213867 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:09.213832    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:09.213968 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:09.213912    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:09.213968 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:09.213960    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:09.566274 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:09.566151    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt" event={"ID":"5de424bb-588e-4a60-a6a3-5bacf8e71f1e","Type":"ContainerStarted","Data":"1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7"}
Sep 09 20:53:09.566274 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:09.566258    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt" event={"ID":"5de424bb-588e-4a60-a6a3-5bacf8e71f1e","Type":"ContainerStarted","Data":"d66afe15f009b7eae787753baa990cab357e0486f0940d4e97f413669b9b592d"}
Sep 09 20:53:09.588329 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:09.588285    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt" podStartSLOduration=1.5882704269999999 podStartE2EDuration="1.588270427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:09.586310899 +0000 UTC m=+1249.736983242" watchObservedRunningTime="2026-09-09 20:53:09.588270427 +0000 UTC m=+1249.738942770"
Sep 09 20:53:09.917996 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:09.917920    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:09.919646 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:09.919624    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:10.016823 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.016796    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:10.117906 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.117879    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:10.401520 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.401492    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:10.697313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.697220    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"]
Sep 09 20:53:10.700743 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.700720    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.702969 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.702934    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-fbe7be47-openshift-default-dockercfg-zdblg\""
Sep 09 20:53:10.719569 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.719545    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"]
Sep 09 20:53:10.748724 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.748692    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:10.810198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.809233    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-data\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.810198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.809316    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-socket\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.810198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.809423    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-credential-socket\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.810198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.809476    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-podinfo\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.810198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.809675    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5d2n\" (UniqueName: \"kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-kube-api-access-p5d2n\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.810198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.809745    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-envoy\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.810198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.809792    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-token\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.810198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.809819    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istiod-ca-cert\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.810198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.809845    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-certs\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911039 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911008    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-socket\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911227 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911063    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-credential-socket\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911227 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911082    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-podinfo\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911227 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911137    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5d2n\" (UniqueName: \"kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-kube-api-access-p5d2n\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911227 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911155    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-envoy\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911401 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911309    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-token\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911401 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911345    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istiod-ca-cert\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911401 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911370    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-certs\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911567 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911451    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-data\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911567 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911457    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-socket\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911567 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911491    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-credential-socket\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911746 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911726    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-data\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.911852 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.911833    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-certs\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.912228 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.912206    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istiod-ca-cert\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.913603 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.913584    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-envoy\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.913701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.913687    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-podinfo\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.919735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.919710    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-token\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.920025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.919946    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5d2n\" (UniqueName: \"kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-kube-api-access-p5d2n\") pod \"e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") " pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:10.924668 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:10.924647    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:11.013912 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.013879    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:11.099730 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.099689    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:11.300318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.300290    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"]
Sep 09 20:53:11.300945 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:53:11.300921    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4b1801e_a3b5_430d_85f9_86a493ce8ca5.slice/crio-141db6a2a45aee3b8a8751519ab524cf8e563339f08f65b75e405d67754e5910 WatchSource:0}: Error finding container 141db6a2a45aee3b8a8751519ab524cf8e563339f08f65b75e405d67754e5910: Status 404 returned error can't find the container with id 141db6a2a45aee3b8a8751519ab524cf8e563339f08f65b75e405d67754e5910
Sep 09 20:53:11.303214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.303152    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:11.303308 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.303245    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:11.303308 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.303273    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:11.581287 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.581198    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" event={"ID":"a4b1801e-a3b5-430d-85f9-86a493ce8ca5","Type":"ContainerStarted","Data":"cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1"}
Sep 09 20:53:11.581287 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.581266    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" event={"ID":"a4b1801e-a3b5-430d-85f9-86a493ce8ca5","Type":"ContainerStarted","Data":"141db6a2a45aee3b8a8751519ab524cf8e563339f08f65b75e405d67754e5910"}
Sep 09 20:53:11.581649 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.581633    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:11.582629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.582607    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:53:11.601238 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:11.601186    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" podStartSLOduration=1.60115317 podStartE2EDuration="1.60115317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:11.600045235 +0000 UTC m=+1251.750717575" watchObservedRunningTime="2026-09-09 20:53:11.60115317 +0000 UTC m=+1251.751825513"
Sep 09 20:53:12.014692 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:12.014660    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:12.019275 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:12.019251    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:12.587605 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:12.587562    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:12.588537 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:12.588520    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:13.294024 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:13.293975    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:13.364843 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:13.364805    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:13.605993 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:13.605899    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:13.695867 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:13.695831    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:14.886328 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:14.886289    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:14.993596 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:14.993557    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:15.342250 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:15.342208    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:15.495381 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:15.495350    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:15.594132 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:15.594040    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:31.990406 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:31.990371    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"]
Sep 09 20:53:32.015198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:32.015155    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:38.398946 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:38.398908    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:43.479759 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:43.479725    2550 patch_prober.go:28] interesting pod/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": read tcp 10.132.0.2:50684->10.132.0.75:15021: read: connection reset by peer" start-of-body=
Sep 09 20:53:43.480109 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:43.479809    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": read tcp 10.132.0.2:50684->10.132.0.75:15021: read: connection reset by peer"
Sep 09 20:53:43.775872 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:43.775771    2550 generic.go:356] "Generic (PLEG): container finished" podID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerID="cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1" exitCode=0
Sep 09 20:53:43.775872 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:43.775848    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" event={"ID":"a4b1801e-a3b5-430d-85f9-86a493ce8ca5","Type":"ContainerDied","Data":"cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1"}
Sep 09 20:53:43.776358 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:43.776341    2550 scope.go:117] "RemoveContainer" containerID="cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1"
Sep 09 20:53:43.777614 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:43.777588    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:43.777677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:43.777654    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:43.777711 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:43.777686    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:44.698630 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:44.698600    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:44.783688 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:44.783657    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" event={"ID":"a4b1801e-a3b5-430d-85f9-86a493ce8ca5","Type":"ContainerStarted","Data":"3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b"}
Sep 09 20:53:45.014539 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:45.014501    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:45.014707 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:45.014680    2550 patch_prober.go:28] interesting pod/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 20:53:45.014753 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:45.014733    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 20:53:46.014715 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:46.014675    2550 patch_prober.go:28] interesting pod/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 20:53:46.015092 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:46.014730    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 20:53:47.010214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.010141    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg" podUID="c22eeea6-2581-4705-a6db-1e17b960fc8b" containerName="main" containerID="cri-o://47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912" gracePeriod=45
Sep 09 20:53:47.014617 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.014589    2550 patch_prober.go:28] interesting pod/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 20:53:47.014727 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.014643    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 20:53:47.312472 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.312448    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:53:47.383152 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383121    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-home\") pod \"c22eeea6-2581-4705-a6db-1e17b960fc8b\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") "
Sep 09 20:53:47.383345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383189    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-dshm\") pod \"c22eeea6-2581-4705-a6db-1e17b960fc8b\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") "
Sep 09 20:53:47.383345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383286    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-model-cache\") pod \"c22eeea6-2581-4705-a6db-1e17b960fc8b\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") "
Sep 09 20:53:47.383345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383314    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22eeea6-2581-4705-a6db-1e17b960fc8b-tls-certs\") pod \"c22eeea6-2581-4705-a6db-1e17b960fc8b\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") "
Sep 09 20:53:47.383345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383337    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-tmp-dir\") pod \"c22eeea6-2581-4705-a6db-1e17b960fc8b\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") "
Sep 09 20:53:47.383522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383377    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-home" (OuterVolumeSpecName: "home") pod "c22eeea6-2581-4705-a6db-1e17b960fc8b" (UID: "c22eeea6-2581-4705-a6db-1e17b960fc8b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:53:47.383522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383393    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdlg4\" (UniqueName: \"kubernetes.io/projected/c22eeea6-2581-4705-a6db-1e17b960fc8b-kube-api-access-sdlg4\") pod \"c22eeea6-2581-4705-a6db-1e17b960fc8b\" (UID: \"c22eeea6-2581-4705-a6db-1e17b960fc8b\") "
Sep 09 20:53:47.383522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383482    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-model-cache" (OuterVolumeSpecName: "model-cache") pod "c22eeea6-2581-4705-a6db-1e17b960fc8b" (UID: "c22eeea6-2581-4705-a6db-1e17b960fc8b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:53:47.383664 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383546    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c22eeea6-2581-4705-a6db-1e17b960fc8b" (UID: "c22eeea6-2581-4705-a6db-1e17b960fc8b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:53:47.383847 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383822    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:53:47.383847 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383846    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:53:47.384038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.383859    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:53:47.385553 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.385522    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22eeea6-2581-4705-a6db-1e17b960fc8b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c22eeea6-2581-4705-a6db-1e17b960fc8b" (UID: "c22eeea6-2581-4705-a6db-1e17b960fc8b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:53:47.385659 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.385573    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-dshm" (OuterVolumeSpecName: "dshm") pod "c22eeea6-2581-4705-a6db-1e17b960fc8b" (UID: "c22eeea6-2581-4705-a6db-1e17b960fc8b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:53:47.385659 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.385606    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22eeea6-2581-4705-a6db-1e17b960fc8b-kube-api-access-sdlg4" (OuterVolumeSpecName: "kube-api-access-sdlg4") pod "c22eeea6-2581-4705-a6db-1e17b960fc8b" (UID: "c22eeea6-2581-4705-a6db-1e17b960fc8b"). InnerVolumeSpecName "kube-api-access-sdlg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:53:47.484777 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.484747    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22eeea6-2581-4705-a6db-1e17b960fc8b-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:53:47.484777 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.484773    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdlg4\" (UniqueName: \"kubernetes.io/projected/c22eeea6-2581-4705-a6db-1e17b960fc8b-kube-api-access-sdlg4\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:53:47.484777 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.484784    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22eeea6-2581-4705-a6db-1e17b960fc8b-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:53:47.801925 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.801881    2550 generic.go:356] "Generic (PLEG): container finished" podID="c22eeea6-2581-4705-a6db-1e17b960fc8b" containerID="47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912" exitCode=0
Sep 09 20:53:47.802125 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.801938    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"
Sep 09 20:53:47.802125 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.801971    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg" event={"ID":"c22eeea6-2581-4705-a6db-1e17b960fc8b","Type":"ContainerDied","Data":"47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912"}
Sep 09 20:53:47.802125 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.802034    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg" event={"ID":"c22eeea6-2581-4705-a6db-1e17b960fc8b","Type":"ContainerDied","Data":"38e9f85e40a1de0b4b9d6df2f48ff95ff6d486c494980478be9efeae8a9ffa76"}
Sep 09 20:53:47.802125 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.802061    2550 scope.go:117] "RemoveContainer" containerID="47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912"
Sep 09 20:53:47.811674 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.811658    2550 scope.go:117] "RemoveContainer" containerID="47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912"
Sep 09 20:53:47.811958 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:53:47.811940    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912\": container with ID starting with 47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912 not found: ID does not exist" containerID="47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912"
Sep 09 20:53:47.812009 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.811966    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912"} err="failed to get container status \"47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912\": rpc error: code = NotFound desc = could not find container \"47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912\": container with ID starting with 47e578b93195bb5ca4906ad53988afade817211b4b59741d752efef50076a912 not found: ID does not exist"
Sep 09 20:53:47.824348 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.824325    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"]
Sep 09 20:53:47.827921 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:47.827902    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c7643f/test-model-c7643f-kserve-c5fdf9459-jvjsg"]
Sep 09 20:53:48.014146 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:48.014111    2550 patch_prober.go:28] interesting pod/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 20:53:48.014332 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:48.014202    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 20:53:48.280671 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:48.280640    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c22eeea6-2581-4705-a6db-1e17b960fc8b" path="/var/lib/kubelet/pods/c22eeea6-2581-4705-a6db-1e17b960fc8b/volumes"
Sep 09 20:53:49.014828 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:49.014790    2550 patch_prober.go:28] interesting pod/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 20:53:49.015250 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:49.014848    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 20:53:50.014682 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:50.014647    2550 patch_prober.go:28] interesting pod/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 20:53:50.014833 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:50.014700    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 20:53:51.018121 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:51.018091    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:51.018535 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:51.018498    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:53:51.019199 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:53:51.019154    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:54:03.155199 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:03.155117    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:04.201843 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.201810    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"]
Sep 09 20:54:04.202313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.202298    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22eeea6-2581-4705-a6db-1e17b960fc8b" containerName="main"
Sep 09 20:54:04.202390 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.202314    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22eeea6-2581-4705-a6db-1e17b960fc8b" containerName="main"
Sep 09 20:54:04.202448 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.202412    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22eeea6-2581-4705-a6db-1e17b960fc8b" containerName="main"
Sep 09 20:54:04.214593 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.214563    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.216768 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.216742    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-933e9d\"/\"distinct-2-w4-933e9d-kserve-self-signed-certs\""
Sep 09 20:54:04.216923 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.216898    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"]
Sep 09 20:54:04.236267 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.236232    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:04.358190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.356794    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcaf2446-4001-4709-bf08-1c2a13170b77-tls-certs\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.358190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.356854    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-dshm\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.358190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.356946    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-tmp-dir\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.358190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.357042    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-home\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.358190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.357069    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b7gd\" (UniqueName: \"kubernetes.io/projected/fcaf2446-4001-4709-bf08-1c2a13170b77-kube-api-access-7b7gd\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.358190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.357143    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-model-cache\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.457794 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.457711    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-model-cache\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.457794 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.457758    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcaf2446-4001-4709-bf08-1c2a13170b77-tls-certs\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.457794 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.457784    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-dshm\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.458020 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.457825    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-tmp-dir\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.458020 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.457886    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-home\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.458020 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.457910    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b7gd\" (UniqueName: \"kubernetes.io/projected/fcaf2446-4001-4709-bf08-1c2a13170b77-kube-api-access-7b7gd\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.458189 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.458144    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-model-cache\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.458290 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.458267    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-tmp-dir\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.458387 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.458367    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-home\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.460123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.460101    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-dshm\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.460400 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.460374    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcaf2446-4001-4709-bf08-1c2a13170b77-tls-certs\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.465947 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.465927    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b7gd\" (UniqueName: \"kubernetes.io/projected/fcaf2446-4001-4709-bf08-1c2a13170b77-kube-api-access-7b7gd\") pod \"distinct-2-w4-933e9d-kserve-5849f96745-6bqcm\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") " pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.525054 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.525016    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:04.797910 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.797876    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"]
Sep 09 20:54:04.798129 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:54:04.798107    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcaf2446_4001_4709_bf08_1c2a13170b77.slice/crio-85df44c238607d782f9ef8b51663f517ebcc60ca770f5f4ad6e6d8406473f9e4 WatchSource:0}: Error finding container 85df44c238607d782f9ef8b51663f517ebcc60ca770f5f4ad6e6d8406473f9e4: Status 404 returned error can't find the container with id 85df44c238607d782f9ef8b51663f517ebcc60ca770f5f4ad6e6d8406473f9e4
Sep 09 20:54:04.903912 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.903883    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm" event={"ID":"fcaf2446-4001-4709-bf08-1c2a13170b77","Type":"ContainerStarted","Data":"fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39"}
Sep 09 20:54:04.904042 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.903923    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm" event={"ID":"fcaf2446-4001-4709-bf08-1c2a13170b77","Type":"ContainerStarted","Data":"85df44c238607d782f9ef8b51663f517ebcc60ca770f5f4ad6e6d8406473f9e4"}
Sep 09 20:54:04.922656 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:04.922604    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm" podStartSLOduration=0.92258983 podStartE2EDuration="922.58983ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:04.920798946 +0000 UTC m=+1305.071471292" watchObservedRunningTime="2026-09-09 20:54:04.92258983 +0000 UTC m=+1305.073262203"
Sep 09 20:54:13.614325 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.614284    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"]
Sep 09 20:54:13.614819 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.614528    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68" podUID="f921fe53-3f2a-4f42-82f2-6fc532a52117" containerName="maas-api" containerID="cri-o://7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4" gracePeriod=30
Sep 09 20:54:13.624691 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.624648    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:13.708679 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.708644    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:13.780719 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.780681    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:13.846507 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.846467    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:13.896839 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.896812    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:54:13.910979 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.910948    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:13.955393 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.953937    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf6qm\" (UniqueName: \"kubernetes.io/projected/f921fe53-3f2a-4f42-82f2-6fc532a52117-kube-api-access-mf6qm\") pod \"f921fe53-3f2a-4f42-82f2-6fc532a52117\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") "
Sep 09 20:54:13.955393 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.954061    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-metrics-certs\") pod \"f921fe53-3f2a-4f42-82f2-6fc532a52117\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") "
Sep 09 20:54:13.955393 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.954106    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f921fe53-3f2a-4f42-82f2-6fc532a52117-openshift-service-ca\") pod \"f921fe53-3f2a-4f42-82f2-6fc532a52117\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") "
Sep 09 20:54:13.962197 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.960981    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f921fe53-3f2a-4f42-82f2-6fc532a52117-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f921fe53-3f2a-4f42-82f2-6fc532a52117" (UID: "f921fe53-3f2a-4f42-82f2-6fc532a52117"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:13.965585 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.963699    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls\") pod \"f921fe53-3f2a-4f42-82f2-6fc532a52117\" (UID: \"f921fe53-3f2a-4f42-82f2-6fc532a52117\") "
Sep 09 20:54:13.965585 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.964344    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f921fe53-3f2a-4f42-82f2-6fc532a52117-openshift-service-ca\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.974761 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.974694    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f921fe53-3f2a-4f42-82f2-6fc532a52117" (UID: "f921fe53-3f2a-4f42-82f2-6fc532a52117"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:54:13.977842 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.977799    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f921fe53-3f2a-4f42-82f2-6fc532a52117-kube-api-access-mf6qm" (OuterVolumeSpecName: "kube-api-access-mf6qm") pod "f921fe53-3f2a-4f42-82f2-6fc532a52117" (UID: "f921fe53-3f2a-4f42-82f2-6fc532a52117"). InnerVolumeSpecName "kube-api-access-mf6qm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:13.978072 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.978033    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f921fe53-3f2a-4f42-82f2-6fc532a52117" (UID: "f921fe53-3f2a-4f42-82f2-6fc532a52117"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:54:14.001187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.999881    2550 generic.go:356] "Generic (PLEG): container finished" podID="f921fe53-3f2a-4f42-82f2-6fc532a52117" containerID="7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4" exitCode=0
Sep 09 20:54:14.001187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:13.999957    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68" event={"ID":"f921fe53-3f2a-4f42-82f2-6fc532a52117","Type":"ContainerDied","Data":"7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4"}
Sep 09 20:54:14.001187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.000049    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68" event={"ID":"f921fe53-3f2a-4f42-82f2-6fc532a52117","Type":"ContainerDied","Data":"16e1d567c106f14d05813c4211c2b3055cf4ac3cf9920e351f1c75f3a54266d8"}
Sep 09 20:54:14.001187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.000083    2550 scope.go:117] "RemoveContainer" containerID="7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4"
Sep 09 20:54:14.001187 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.000125    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"
Sep 09 20:54:14.015737 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.015678    2550 scope.go:117] "RemoveContainer" containerID="7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4"
Sep 09 20:54:14.016195 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:54:14.016085    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4\": container with ID starting with 7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4 not found: ID does not exist" containerID="7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4"
Sep 09 20:54:14.016195 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.016130    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4"} err="failed to get container status \"7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4\": rpc error: code = NotFound desc = could not find container \"7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4\": container with ID starting with 7b5d9a3dc30a5490323a198ff0453951eca4665147204918571274a0ee0acac4 not found: ID does not exist"
Sep 09 20:54:14.036545 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.035973    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"]
Sep 09 20:54:14.042629 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.042596    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c7643f-587478dbb-w7m68"]
Sep 09 20:54:14.065185 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.065125    2550 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-metrics-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:14.065185 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.065183    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f921fe53-3f2a-4f42-82f2-6fc532a52117-maas-api-tls\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:14.065185 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.065198    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf6qm\" (UniqueName: \"kubernetes.io/projected/f921fe53-3f2a-4f42-82f2-6fc532a52117-kube-api-access-mf6qm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:14.197880 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.197789    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:14.279835 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.279794    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f921fe53-3f2a-4f42-82f2-6fc532a52117" path="/var/lib/kubelet/pods/f921fe53-3f2a-4f42-82f2-6fc532a52117/volumes"
Sep 09 20:54:14.525371 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.525334    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:14.525577 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.525392    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:14.541251 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:14.541223    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:15.018303 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:15.018272    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 20:54:17.407404 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:17.407372    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:17.443800 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:17.443759    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"]
Sep 09 20:54:17.444004 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:17.443977    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt" podUID="5de424bb-588e-4a60-a6a3-5bacf8e71f1e" containerName="istio-proxy" containerID="cri-o://1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7" gracePeriod=30
Sep 09 20:54:17.481635 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:17.481608    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:17.724008 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:17.723979    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:17.883180 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:17.883122    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:20.023498 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.023430    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:20.124297 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.124264    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:20.260946 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.260915    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"]
Sep 09 20:54:20.261190 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.261137    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy" containerID="cri-o://61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678" gracePeriod=30
Sep 09 20:54:20.287989 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.287929    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:20.612713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.612633    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"]
Sep 09 20:54:20.613059 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.613045    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f921fe53-3f2a-4f42-82f2-6fc532a52117" containerName="maas-api"
Sep 09 20:54:20.613109 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.613061    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f921fe53-3f2a-4f42-82f2-6fc532a52117" containerName="maas-api"
Sep 09 20:54:20.613145 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.613138    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f921fe53-3f2a-4f42-82f2-6fc532a52117" containerName="maas-api"
Sep 09 20:54:20.616562 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.616539    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"
Sep 09 20:54:20.618566 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.618537    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tsks9\""
Sep 09 20:54:20.624017 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.623992    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"]
Sep 09 20:54:20.728424 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.728393    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:20.735857 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.735832    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8a6968-2c7a-4ec2-8fe2-51f488142987-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl\" (UID: \"9e8a6968-2c7a-4ec2-8fe2-51f488142987\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"
Sep 09 20:54:20.837485 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.837447    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8a6968-2c7a-4ec2-8fe2-51f488142987-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl\" (UID: \"9e8a6968-2c7a-4ec2-8fe2-51f488142987\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"
Sep 09 20:54:20.838034 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.838013    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8a6968-2c7a-4ec2-8fe2-51f488142987-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl\" (UID: \"9e8a6968-2c7a-4ec2-8fe2-51f488142987\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"
Sep 09 20:54:20.929387 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:20.929321    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"
Sep 09 20:54:21.203806 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:21.203777    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"]
Sep 09 20:54:21.204729 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:54:21.204703    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8a6968_2c7a_4ec2_8fe2_51f488142987.slice/crio-48e4212904427d5d5d6130ba968239b1929bdb45fd71860a29b8ce75b795d18f WatchSource:0}: Error finding container 48e4212904427d5d5d6130ba968239b1929bdb45fd71860a29b8ce75b795d18f: Status 404 returned error can't find the container with id 48e4212904427d5d5d6130ba968239b1929bdb45fd71860a29b8ce75b795d18f
Sep 09 20:54:21.206672 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:21.206655    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:54:22.049463 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.049428    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl" event={"ID":"9e8a6968-2c7a-4ec2-8fe2-51f488142987","Type":"ContainerStarted","Data":"48e4212904427d5d5d6130ba968239b1929bdb45fd71860a29b8ce75b795d18f"}
Sep 09 20:54:22.193624 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.193592    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:22.721481 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.721455    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:54:22.860308 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.860269    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-socket\") pod \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") "
Sep 09 20:54:22.860476 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.860331    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-token\") pod \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") "
Sep 09 20:54:22.860476 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.860387    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-podinfo\") pod \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") "
Sep 09 20:54:22.860476 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.860428    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-envoy\") pod \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") "
Sep 09 20:54:22.860653 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.860490    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fqcm\" (UniqueName: \"kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-kube-api-access-2fqcm\") pod \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") "
Sep 09 20:54:22.860653 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.860605    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-data\") pod \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") "
Sep 09 20:54:22.860844 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.860658    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-credential-socket\") pod \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") "
Sep 09 20:54:22.860844 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.860685    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istiod-ca-cert\") pod \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") "
Sep 09 20:54:22.860844 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.860733    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-certs\") pod \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\" (UID: \"5de424bb-588e-4a60-a6a3-5bacf8e71f1e\") "
Sep 09 20:54:22.860844 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.860597    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5de424bb-588e-4a60-a6a3-5bacf8e71f1e" (UID: "5de424bb-588e-4a60-a6a3-5bacf8e71f1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:22.861067 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.861011    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-data" (OuterVolumeSpecName: "istio-data") pod "5de424bb-588e-4a60-a6a3-5bacf8e71f1e" (UID: "5de424bb-588e-4a60-a6a3-5bacf8e71f1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:22.861449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.861221    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5de424bb-588e-4a60-a6a3-5bacf8e71f1e" (UID: "5de424bb-588e-4a60-a6a3-5bacf8e71f1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:22.861449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.861250    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:22.861449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.861275    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:22.861449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.861368    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5de424bb-588e-4a60-a6a3-5bacf8e71f1e" (UID: "5de424bb-588e-4a60-a6a3-5bacf8e71f1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:22.861808 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.861544    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5de424bb-588e-4a60-a6a3-5bacf8e71f1e" (UID: "5de424bb-588e-4a60-a6a3-5bacf8e71f1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:22.863114 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.863090    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5de424bb-588e-4a60-a6a3-5bacf8e71f1e" (UID: "5de424bb-588e-4a60-a6a3-5bacf8e71f1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:54:22.863245 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.863118    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-kube-api-access-2fqcm" (OuterVolumeSpecName: "kube-api-access-2fqcm") pod "5de424bb-588e-4a60-a6a3-5bacf8e71f1e" (UID: "5de424bb-588e-4a60-a6a3-5bacf8e71f1e"). InnerVolumeSpecName "kube-api-access-2fqcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:22.863245 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.863218    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5de424bb-588e-4a60-a6a3-5bacf8e71f1e" (UID: "5de424bb-588e-4a60-a6a3-5bacf8e71f1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:22.863816 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.863798    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5de424bb-588e-4a60-a6a3-5bacf8e71f1e" (UID: "5de424bb-588e-4a60-a6a3-5bacf8e71f1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:22.894606 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.894569    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:22.962106 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.962061    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:22.962106 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.962099    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:22.962106 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.962113    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:22.962417 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.962126    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:22.962417 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.962139    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:22.962417 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.962151    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:22.962417 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:22.962183    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fqcm\" (UniqueName: \"kubernetes.io/projected/5de424bb-588e-4a60-a6a3-5bacf8e71f1e-kube-api-access-2fqcm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:23.058140 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:23.058051    2550 generic.go:356] "Generic (PLEG): container finished" podID="5de424bb-588e-4a60-a6a3-5bacf8e71f1e" containerID="1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7" exitCode=0
Sep 09 20:54:23.058140 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:23.058120    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"
Sep 09 20:54:23.058140 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:23.058129    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt" event={"ID":"5de424bb-588e-4a60-a6a3-5bacf8e71f1e","Type":"ContainerDied","Data":"1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7"}
Sep 09 20:54:23.058444 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:23.058220    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt" event={"ID":"5de424bb-588e-4a60-a6a3-5bacf8e71f1e","Type":"ContainerDied","Data":"d66afe15f009b7eae787753baa990cab357e0486f0940d4e97f413669b9b592d"}
Sep 09 20:54:23.058444 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:23.058249    2550 scope.go:117] "RemoveContainer" containerID="1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7"
Sep 09 20:54:23.084156 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:23.084124    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"]
Sep 09 20:54:23.087483 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:23.087459    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-a214cb-openshift-default-95b87f746-5x8lt"]
Sep 09 20:54:23.612480 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:23.612263    2550 scope.go:117] "RemoveContainer" containerID="1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7"
Sep 09 20:54:23.612702 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:54:23.612683    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7\": container with ID starting with 1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7 not found: ID does not exist" containerID="1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7"
Sep 09 20:54:23.612776 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:23.612713    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7"} err="failed to get container status \"1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7\": rpc error: code = NotFound desc = could not find container \"1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7\": container with ID starting with 1f9c751a5d6fdb90c512590a7bcab0f61a3302c80294570786670ad5a59834f7 not found: ID does not exist"
Sep 09 20:54:24.066556 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:24.066515    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl" event={"ID":"9e8a6968-2c7a-4ec2-8fe2-51f488142987","Type":"ContainerStarted","Data":"8db306d87c170eb2da416209273c833c84187d4d4b736b82da22b147997ca952"}
Sep 09 20:54:24.081689 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:24.081645    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl" podStartSLOduration=1.613863965 podStartE2EDuration="4.081632081s" totalImagesPullingTime="2.467768116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:24.080103867 +0000 UTC m=+1324.230776210" watchObservedRunningTime="2026-09-09 20:54:24.081632081 +0000 UTC m=+1324.232304424"
Sep 09 20:54:24.280686 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:24.280656    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5de424bb-588e-4a60-a6a3-5bacf8e71f1e" path="/var/lib/kubelet/pods/5de424bb-588e-4a60-a6a3-5bacf8e71f1e/volumes"
Sep 09 20:54:25.074066 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.074029    2550 generic.go:356] "Generic (PLEG): container finished" podID="9e8a6968-2c7a-4ec2-8fe2-51f488142987" containerID="8db306d87c170eb2da416209273c833c84187d4d4b736b82da22b147997ca952" exitCode=0
Sep 09 20:54:25.074459 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.074082    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl" event={"ID":"9e8a6968-2c7a-4ec2-8fe2-51f488142987","Type":"ContainerDied","Data":"8db306d87c170eb2da416209273c833c84187d4d4b736b82da22b147997ca952"}
Sep 09 20:54:25.524244 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.524221    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:54:25.587558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587526    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-data\") pod \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") "
Sep 09 20:54:25.587675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587580    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-envoy\") pod \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") "
Sep 09 20:54:25.587675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587617    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-token\") pod \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") "
Sep 09 20:54:25.587675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587639    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-credential-socket\") pod \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") "
Sep 09 20:54:25.587842 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587683    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-certs\") pod \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") "
Sep 09 20:54:25.587842 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587727    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-podinfo\") pod \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") "
Sep 09 20:54:25.587842 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587766    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-socket\") pod \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") "
Sep 09 20:54:25.587842 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587772    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" (UID: "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:25.588033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587870    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istiod-ca-cert\") pod \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") "
Sep 09 20:54:25.588033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587915    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8plfz\" (UniqueName: \"kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-kube-api-access-8plfz\") pod \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\" (UID: \"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37\") "
Sep 09 20:54:25.588033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.587951    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" (UID: "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:25.588033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.588009    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" (UID: "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:25.588284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.588261    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" (UID: "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:25.588357 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.588328    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" (UID: "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:25.588419 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.588360    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:25.588419 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.588372    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:25.588419 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.588381    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:25.588419 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.588389    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:25.589834 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.589808    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" (UID: "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:25.589961 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.589840    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" (UID: "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:25.590020 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.589966    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" (UID: "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:54:25.590020 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.589984    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-kube-api-access-8plfz" (OuterVolumeSpecName: "kube-api-access-8plfz") pod "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" (UID: "9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37"). InnerVolumeSpecName "kube-api-access-8plfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:25.689713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.689652    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:25.689713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.689672    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8plfz\" (UniqueName: \"kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-kube-api-access-8plfz\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:25.689713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.689694    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:25.689713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.689702    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:25.689713 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:25.689711    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:26.081872 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.081841    2550 generic.go:356] "Generic (PLEG): container finished" podID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerID="61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678" exitCode=0
Sep 09 20:54:26.082366 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.081887    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"
Sep 09 20:54:26.082366 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.081904    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" event={"ID":"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37","Type":"ContainerDied","Data":"61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678"}
Sep 09 20:54:26.082366 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.081976    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs" event={"ID":"9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37","Type":"ContainerDied","Data":"9f3afb6e942c8e3d5e61dd11ae9bf7f5daa3654a66ace8d4c0cc188d22971768"}
Sep 09 20:54:26.082366 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.082011    2550 scope.go:117] "RemoveContainer" containerID="61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678"
Sep 09 20:54:26.092027 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.092005    2550 scope.go:117] "RemoveContainer" containerID="ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0"
Sep 09 20:54:26.102093 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.102075    2550 scope.go:117] "RemoveContainer" containerID="61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678"
Sep 09 20:54:26.102433 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:54:26.102406    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678\": container with ID starting with 61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678 not found: ID does not exist" containerID="61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678"
Sep 09 20:54:26.102522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.102437    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678"} err="failed to get container status \"61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678\": rpc error: code = NotFound desc = could not find container \"61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678\": container with ID starting with 61f45344c087f47b51eae16f733ac1d8e97eab0cbd127bd8fb8ee0d6daa6f678 not found: ID does not exist"
Sep 09 20:54:26.102522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.102455    2550 scope.go:117] "RemoveContainer" containerID="ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0"
Sep 09 20:54:26.102706 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:54:26.102692    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0\": container with ID starting with ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0 not found: ID does not exist" containerID="ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0"
Sep 09 20:54:26.102753 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.102710    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0"} err="failed to get container status \"ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0\": rpc error: code = NotFound desc = could not find container \"ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0\": container with ID starting with ddfa2b934f7b0465fb2097d54e8c2931bf14c1ca8355115156272a8658c74bb0 not found: ID does not exist"
Sep 09 20:54:26.106905 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.106882    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"]
Sep 09 20:54:26.114723 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.114702    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c7643f-openshift-default-7985ccd8c6-n6vhs"]
Sep 09 20:54:26.214522 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.214498    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"
Sep 09 20:54:26.280241 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.280134    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" path="/var/lib/kubelet/pods/9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37/volumes"
Sep 09 20:54:26.295563 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.295534    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8a6968-2c7a-4ec2-8fe2-51f488142987-maas-api-service-ca\") pod \"9e8a6968-2c7a-4ec2-8fe2-51f488142987\" (UID: \"9e8a6968-2c7a-4ec2-8fe2-51f488142987\") "
Sep 09 20:54:26.295839 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.295819    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8a6968-2c7a-4ec2-8fe2-51f488142987-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e8a6968-2c7a-4ec2-8fe2-51f488142987" (UID: "9e8a6968-2c7a-4ec2-8fe2-51f488142987"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:26.396432 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:26.396352    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8a6968-2c7a-4ec2-8fe2-51f488142987-maas-api-service-ca\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:54:27.089784 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:27.089753    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"
Sep 09 20:54:27.089784 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:27.089759    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl" event={"ID":"9e8a6968-2c7a-4ec2-8fe2-51f488142987","Type":"ContainerDied","Data":"48e4212904427d5d5d6130ba968239b1929bdb45fd71860a29b8ce75b795d18f"}
Sep 09 20:54:27.090283 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:27.089813    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48e4212904427d5d5d6130ba968239b1929bdb45fd71860a29b8ce75b795d18f"
Sep 09 20:54:31.263929 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:31.263813    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:31.499597 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:31.499565    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"]
Sep 09 20:54:31.503664 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:31.503635    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2dd30-jckrl"]
Sep 09 20:54:32.287650 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:32.287610    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e8a6968-2c7a-4ec2-8fe2-51f488142987" path="/var/lib/kubelet/pods/9e8a6968-2c7a-4ec2-8fe2-51f488142987/volumes"
Sep 09 20:54:33.597019 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:33.596982    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:36.928185 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:36.928127    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:42.972662 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:42.972555    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:43.262691 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:43.262601    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:43.795970 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:43.795931    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:44.123575 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:44.123483    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:44.748069 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:44.748031    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:45.357283 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:45.357241    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:47.476755 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:47.476722    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:47.611602 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:47.611565    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:48.231308 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:48.231268    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:49.115881 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.115851    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:49.200735 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.200692    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:49.572076 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.572036    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:49.754497 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.754457    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:49.850198 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.850061    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:49.881787 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.881755    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"]
Sep 09 20:54:49.882153 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.882142    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5de424bb-588e-4a60-a6a3-5bacf8e71f1e" containerName="istio-proxy"
Sep 09 20:54:49.882234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.882155    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5de424bb-588e-4a60-a6a3-5bacf8e71f1e" containerName="istio-proxy"
Sep 09 20:54:49.882234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.882179    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8a6968-2c7a-4ec2-8fe2-51f488142987" containerName="revoke-keys"
Sep 09 20:54:49.882234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.882188    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8a6968-2c7a-4ec2-8fe2-51f488142987" containerName="revoke-keys"
Sep 09 20:54:49.882234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.882201    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy"
Sep 09 20:54:49.882234 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.882209    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy"
Sep 09 20:54:49.882397 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.882317    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8a6968-2c7a-4ec2-8fe2-51f488142987" containerName="revoke-keys"
Sep 09 20:54:49.882397 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.882328    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5de424bb-588e-4a60-a6a3-5bacf8e71f1e" containerName="istio-proxy"
Sep 09 20:54:49.882397 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.882334    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy"
Sep 09 20:54:49.886556 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.886541    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 20:54:49.889648 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.889629    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-cb82f2\""
Sep 09 20:54:49.889744 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.889646    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-cb82f2-dockercfg-6vqhz\""
Sep 09 20:54:49.902570 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.902543    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"]
Sep 09 20:54:49.923318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.923280    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5jzc\" (UniqueName: \"kubernetes.io/projected/ef417d3c-c927-4700-9789-05cb639a76d1-kube-api-access-b5jzc\") pod \"payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9\" (UID: \"ef417d3c-c927-4700-9789-05cb639a76d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 20:54:49.923472 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:49.923353    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef417d3c-c927-4700-9789-05cb639a76d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9\" (UID: \"ef417d3c-c927-4700-9789-05cb639a76d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 20:54:50.025222 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:50.025181    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5jzc\" (UniqueName: \"kubernetes.io/projected/ef417d3c-c927-4700-9789-05cb639a76d1-kube-api-access-b5jzc\") pod \"payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9\" (UID: \"ef417d3c-c927-4700-9789-05cb639a76d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 20:54:50.025416 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:50.025258    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef417d3c-c927-4700-9789-05cb639a76d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9\" (UID: \"ef417d3c-c927-4700-9789-05cb639a76d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 20:54:50.027922 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:50.027889    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef417d3c-c927-4700-9789-05cb639a76d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9\" (UID: \"ef417d3c-c927-4700-9789-05cb639a76d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 20:54:50.037186 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:50.037103    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5jzc\" (UniqueName: \"kubernetes.io/projected/ef417d3c-c927-4700-9789-05cb639a76d1-kube-api-access-b5jzc\") pod \"payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9\" (UID: \"ef417d3c-c927-4700-9789-05cb639a76d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 20:54:50.197283 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:50.197195    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 20:54:50.476693 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:50.476616    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"]
Sep 09 20:54:51.238025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:51.237987    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9" event={"ID":"ef417d3c-c927-4700-9789-05cb639a76d1","Type":"ContainerStarted","Data":"2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e"}
Sep 09 20:54:51.238519 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:51.238052    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9" event={"ID":"ef417d3c-c927-4700-9789-05cb639a76d1","Type":"ContainerStarted","Data":"7c5ba1a5324145d6c8e1a285b6eda51caf316b8446dbd8af9b94a8470c0e9847"}
Sep 09 20:54:51.238519 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:51.238084    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 20:54:51.254790 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:51.254740    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9" podStartSLOduration=2.254723793 podStartE2EDuration="2.254723793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:51.253299492 +0000 UTC m=+1351.403971834" watchObservedRunningTime="2026-09-09 20:54:51.254723793 +0000 UTC m=+1351.405396135"
Sep 09 20:54:52.778865 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.778831    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"]
Sep 09 20:54:52.779296 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.779280    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy"
Sep 09 20:54:52.779351 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.779298    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy"
Sep 09 20:54:52.779387 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.779359    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f61cfc8-70cc-4f5b-bb3e-19bcc5671c37" containerName="istio-proxy"
Sep 09 20:54:52.782501 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.782486    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.784972 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.784948    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fbe7be47\"/\"e2e-lifecycle-model-fbe7be47-kserve-self-signed-certs\""
Sep 09 20:54:52.784972 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.784949    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fbe7be47\"/\"kube-root-ca.crt\""
Sep 09 20:54:52.785158 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.784949    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fbe7be47\"/\"openshift-service-ca.crt\""
Sep 09 20:54:52.785158 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.785001    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fbe7be47\"/\"default-dockercfg-d7n5p\""
Sep 09 20:54:52.792521 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.792499    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"]
Sep 09 20:54:52.856044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.856004    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-model-cache\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.856221 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.856064    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tls-certs\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.856221 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.856108    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7dgw\" (UniqueName: \"kubernetes.io/projected/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-kube-api-access-f7dgw\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.856221 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.856133    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tmp-dir\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.856221 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.856201    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-home\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.856221 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.856217    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-dshm\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.957570 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.957540    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-model-cache\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.957712 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.957587    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tls-certs\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.957712 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.957609    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7dgw\" (UniqueName: \"kubernetes.io/projected/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-kube-api-access-f7dgw\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.957712 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.957634    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tmp-dir\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.957712 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.957672    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-home\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.957712 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.957688    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-dshm\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.958013 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.957993    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-model-cache\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.958074 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.958014    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tmp-dir\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.958123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.958084    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-home\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.959938 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.959915    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-dshm\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.960138 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.960119    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tls-certs\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:52.965106 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:52.965084    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7dgw\" (UniqueName: \"kubernetes.io/projected/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-kube-api-access-f7dgw\") pod \"e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") " pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:53.080917 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:53.080887    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:53.092085 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:53.092060    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:54:53.371056 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:53.371031    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"]
Sep 09 20:54:53.372028 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:54:53.371997    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63eee0a0_9016_434b_b4cc_9d6205bc4c5f.slice/crio-ac4c71b6c43d2714f7fe90adce25dc371e120d0c9034aa667325e83058e86ee4 WatchSource:0}: Error finding container ac4c71b6c43d2714f7fe90adce25dc371e120d0c9034aa667325e83058e86ee4: Status 404 returned error can't find the container with id ac4c71b6c43d2714f7fe90adce25dc371e120d0c9034aa667325e83058e86ee4
Sep 09 20:54:54.257802 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:54.257763    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6" event={"ID":"63eee0a0-9016-434b-b4cc-9d6205bc4c5f","Type":"ContainerStarted","Data":"89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5"}
Sep 09 20:54:54.258157 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:54.257813    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6" event={"ID":"63eee0a0-9016-434b-b4cc-9d6205bc4c5f","Type":"ContainerStarted","Data":"ac4c71b6c43d2714f7fe90adce25dc371e120d0c9034aa667325e83058e86ee4"}
Sep 09 20:54:54.285426 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:54.285373    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6" podStartSLOduration=2.285357799 podStartE2EDuration="2.285357799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:54.283229173 +0000 UTC m=+1354.433901517" watchObservedRunningTime="2026-09-09 20:54:54.285357799 +0000 UTC m=+1354.436030142"
Sep 09 20:54:58.194559 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:54:58.194523    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:00.034755 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:00.034705    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:02.247739 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:02.247708    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 20:55:03.092449 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:03.092415    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:55:03.092610 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:03.092463    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:55:03.105114 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:03.105083    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:55:03.327575 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:03.327548    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:55:05.541025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:05.540961    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:12.438084 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:12.438048    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:12.791615 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:12.791569    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:13.139764 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:13.139675    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:13.369869 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:13.369838    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:16.727221 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:16.727156    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:16.889926 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:16.889886    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:17.937110 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:17.937067    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:18.036913 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.036869    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:18.139826 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.139782    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:18.534708 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.534668    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:18.651278 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.651240    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:18.721916 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.721885    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"]
Sep 09 20:55:18.727428 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.727409    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 20:55:18.730864 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.730839    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-a2fb5a\""
Sep 09 20:55:18.730995 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.730972    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-a2fb5a-dockercfg-6bx44\""
Sep 09 20:55:18.737792 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.737765    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"]
Sep 09 20:55:18.833268 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.833183    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"]
Sep 09 20:55:18.837678 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.837657    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 20:55:18.858803 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.858768    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"]
Sep 09 20:55:18.927868 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.927828    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69effffe-d680-426a-9518-031affe8ad54-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc\" (UID: \"69effffe-d680-426a-9518-031affe8ad54\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 20:55:18.927868 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:18.927862    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxvzh\" (UniqueName: \"kubernetes.io/projected/69effffe-d680-426a-9518-031affe8ad54-kube-api-access-qxvzh\") pod \"payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc\" (UID: \"69effffe-d680-426a-9518-031affe8ad54\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 20:55:19.029344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.029311    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk\" (UID: \"c15e5119-cb46-486a-8b6e-d5a2db44b9ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 20:55:19.029701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.029359    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-289k6\" (UniqueName: \"kubernetes.io/projected/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-kube-api-access-289k6\") pod \"payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk\" (UID: \"c15e5119-cb46-486a-8b6e-d5a2db44b9ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 20:55:19.029701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.029538    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69effffe-d680-426a-9518-031affe8ad54-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc\" (UID: \"69effffe-d680-426a-9518-031affe8ad54\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 20:55:19.029701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.029569    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxvzh\" (UniqueName: \"kubernetes.io/projected/69effffe-d680-426a-9518-031affe8ad54-kube-api-access-qxvzh\") pod \"payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc\" (UID: \"69effffe-d680-426a-9518-031affe8ad54\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 20:55:19.030243 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.030219    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69effffe-d680-426a-9518-031affe8ad54-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc\" (UID: \"69effffe-d680-426a-9518-031affe8ad54\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 20:55:19.038884 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.038861    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxvzh\" (UniqueName: \"kubernetes.io/projected/69effffe-d680-426a-9518-031affe8ad54-kube-api-access-qxvzh\") pod \"payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc\" (UID: \"69effffe-d680-426a-9518-031affe8ad54\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 20:55:19.041401 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.041381    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 20:55:19.131014 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.130616    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk\" (UID: \"c15e5119-cb46-486a-8b6e-d5a2db44b9ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 20:55:19.131014 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.130678    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-289k6\" (UniqueName: \"kubernetes.io/projected/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-kube-api-access-289k6\") pod \"payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk\" (UID: \"c15e5119-cb46-486a-8b6e-d5a2db44b9ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 20:55:19.131787 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.131473    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk\" (UID: \"c15e5119-cb46-486a-8b6e-d5a2db44b9ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 20:55:19.141266 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.141218    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-289k6\" (UniqueName: \"kubernetes.io/projected/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-kube-api-access-289k6\") pod \"payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk\" (UID: \"c15e5119-cb46-486a-8b6e-d5a2db44b9ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 20:55:19.159757 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.159729    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 20:55:19.319977 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.319950    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"]
Sep 09 20:55:19.320786 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:55:19.320759    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69effffe_d680_426a_9518_031affe8ad54.slice/crio-27fc4f4a0fa4326f4f5c3a0579d6220f150721c5e6be3d61fe82502ac74c1397 WatchSource:0}: Error finding container 27fc4f4a0fa4326f4f5c3a0579d6220f150721c5e6be3d61fe82502ac74c1397: Status 404 returned error can't find the container with id 27fc4f4a0fa4326f4f5c3a0579d6220f150721c5e6be3d61fe82502ac74c1397
Sep 09 20:55:19.413986 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.413926    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc" event={"ID":"69effffe-d680-426a-9518-031affe8ad54","Type":"ContainerStarted","Data":"25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c"}
Sep 09 20:55:19.414143 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.414007    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 20:55:19.414143 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.414032    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc" event={"ID":"69effffe-d680-426a-9518-031affe8ad54","Type":"ContainerStarted","Data":"27fc4f4a0fa4326f4f5c3a0579d6220f150721c5e6be3d61fe82502ac74c1397"}
Sep 09 20:55:19.432253 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.432197    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc" podStartSLOduration=1.432151457 podStartE2EDuration="1.432151457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:19.431320761 +0000 UTC m=+1379.581993104" watchObservedRunningTime="2026-09-09 20:55:19.432151457 +0000 UTC m=+1379.582823801"
Sep 09 20:55:19.439452 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:55:19.439425    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc15e5119_cb46_486a_8b6e_d5a2db44b9ef.slice/crio-8eebc7165fba2d3a7a718cde24a605917466d17fe8b03c4e379b251206893179 WatchSource:0}: Error finding container 8eebc7165fba2d3a7a718cde24a605917466d17fe8b03c4e379b251206893179: Status 404 returned error can't find the container with id 8eebc7165fba2d3a7a718cde24a605917466d17fe8b03c4e379b251206893179
Sep 09 20:55:19.439665 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:19.439644    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"]
Sep 09 20:55:20.422697 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:20.422648    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk" event={"ID":"c15e5119-cb46-486a-8b6e-d5a2db44b9ef","Type":"ContainerStarted","Data":"94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72"}
Sep 09 20:55:20.423146 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:20.422721    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk" event={"ID":"c15e5119-cb46-486a-8b6e-d5a2db44b9ef","Type":"ContainerStarted","Data":"8eebc7165fba2d3a7a718cde24a605917466d17fe8b03c4e379b251206893179"}
Sep 09 20:55:20.423146 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:20.422922    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 20:55:20.442622 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:20.442565    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk" podStartSLOduration=2.442541433 podStartE2EDuration="2.442541433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:20.440591718 +0000 UTC m=+1380.591264155" watchObservedRunningTime="2026-09-09 20:55:20.442541433 +0000 UTC m=+1380.593213780"
Sep 09 20:55:30.425078 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:30.425043    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 20:55:31.432249 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:31.432216    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 20:55:33.153649 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:33.153613    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:41.560537 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:55:41.560496    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:01.330618 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:01.330575    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:07.519739 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:07.519688    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:13.319272 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:13.319181    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:16.698763 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:16.698727    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:16.779551 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:16.779483    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:16.844866 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:16.844815    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:16.938009 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:16.937959    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:17.031675 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:17.031617    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:19.983479 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:19.983446    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:24.964912 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:24.964873    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"]
Sep 09 20:56:25.015237 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:25.015198    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:27.339327 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.339292    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"]
Sep 09 20:56:27.342977 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.342959    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.346576 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.346553    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-cb82f2\"/\"kube-root-ca.crt\""
Sep 09 20:56:27.346576 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.346568    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-cb82f2\"/\"openshift-service-ca.crt\""
Sep 09 20:56:27.346751 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.346561    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-cb82f2\"/\"auth-test-model-cb82f2-kserve-self-signed-certs\""
Sep 09 20:56:27.346751 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.346603    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-cb82f2\"/\"default-dockercfg-bjtcp\""
Sep 09 20:56:27.359112 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.359085    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"]
Sep 09 20:56:27.430820 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.430776    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-model-cache\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.431003 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.430831    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19ee405c-6630-4209-813f-7505eb1590a1-tls-certs\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.431003 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.430941    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-tmp-dir\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.431093 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.431022    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvgjf\" (UniqueName: \"kubernetes.io/projected/19ee405c-6630-4209-813f-7505eb1590a1-kube-api-access-jvgjf\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.431145 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.431092    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-dshm\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.431145 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.431121    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-home\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.532318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.532276    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-model-cache\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.532497 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.532326    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19ee405c-6630-4209-813f-7505eb1590a1-tls-certs\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.532497 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.532378    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-tmp-dir\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.532497 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.532433    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvgjf\" (UniqueName: \"kubernetes.io/projected/19ee405c-6630-4209-813f-7505eb1590a1-kube-api-access-jvgjf\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.532671 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.532501    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-dshm\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.532671 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.532527    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-home\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.532773 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.532729    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-model-cache\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.533025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.533000    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-tmp-dir\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.533149 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.533127    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-home\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.535103 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.535079    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-dshm\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.535451 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.535433    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19ee405c-6630-4209-813f-7505eb1590a1-tls-certs\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.551363 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.551339    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvgjf\" (UniqueName: \"kubernetes.io/projected/19ee405c-6630-4209-813f-7505eb1590a1-kube-api-access-jvgjf\") pod \"auth-test-model-cb82f2-kserve-785d78d47f-s4zq4\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.653019 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.652945    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:27.826305 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.826265    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:27.938428 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:27.938346    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"]
Sep 09 20:56:27.943352 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:56:27.943323    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19ee405c_6630_4209_813f_7505eb1590a1.slice/crio-c23cb49e7764a3631b599b1f612ec16132976022276c63f44b980c2c94fe9871 WatchSource:0}: Error finding container c23cb49e7764a3631b599b1f612ec16132976022276c63f44b980c2c94fe9871: Status 404 returned error can't find the container with id c23cb49e7764a3631b599b1f612ec16132976022276c63f44b980c2c94fe9871
Sep 09 20:56:28.845358 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:28.845324    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4" event={"ID":"19ee405c-6630-4209-813f-7505eb1590a1","Type":"ContainerStarted","Data":"72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886"}
Sep 09 20:56:28.845358 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:28.845362    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4" event={"ID":"19ee405c-6630-4209-813f-7505eb1590a1","Type":"ContainerStarted","Data":"c23cb49e7764a3631b599b1f612ec16132976022276c63f44b980c2c94fe9871"}
Sep 09 20:56:28.870557 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:28.870506    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4" podStartSLOduration=1.8704883410000002 podStartE2EDuration="1.870488341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:28.868453359 +0000 UTC m=+1449.019125702" watchObservedRunningTime="2026-09-09 20:56:28.870488341 +0000 UTC m=+1449.021160716"
Sep 09 20:56:32.881101 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:32.881050    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:37.653385 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:37.653354    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:37.653774 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:37.653604    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:37.665897 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:37.665874    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:37.910521 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:37.910437    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 20:56:39.987655 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:39.987616    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6" podUID="63eee0a0-9016-434b-b4cc-9d6205bc4c5f" containerName="main" containerID="cri-o://89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5" gracePeriod=45
Sep 09 20:56:40.239214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.239147    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:56:40.361985 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.361953    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tls-certs\") pod \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") "
Sep 09 20:56:40.362192 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.361995    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-dshm\") pod \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") "
Sep 09 20:56:40.362192 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.362126    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-home\") pod \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") "
Sep 09 20:56:40.362344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.362210    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-model-cache\") pod \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") "
Sep 09 20:56:40.362344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.362263    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tmp-dir\") pod \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") "
Sep 09 20:56:40.362344 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.362307    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7dgw\" (UniqueName: \"kubernetes.io/projected/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-kube-api-access-f7dgw\") pod \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\" (UID: \"63eee0a0-9016-434b-b4cc-9d6205bc4c5f\") "
Sep 09 20:56:40.362491 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.362358    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-home" (OuterVolumeSpecName: "home") pod "63eee0a0-9016-434b-b4cc-9d6205bc4c5f" (UID: "63eee0a0-9016-434b-b4cc-9d6205bc4c5f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:40.362491 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.362444    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-model-cache" (OuterVolumeSpecName: "model-cache") pod "63eee0a0-9016-434b-b4cc-9d6205bc4c5f" (UID: "63eee0a0-9016-434b-b4cc-9d6205bc4c5f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:40.362585 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.362563    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "63eee0a0-9016-434b-b4cc-9d6205bc4c5f" (UID: "63eee0a0-9016-434b-b4cc-9d6205bc4c5f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:40.362789 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.362770    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:56:40.362858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.362794    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:56:40.362858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.362810    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:56:40.364325 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.364299    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "63eee0a0-9016-434b-b4cc-9d6205bc4c5f" (UID: "63eee0a0-9016-434b-b4cc-9d6205bc4c5f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:56:40.364422 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.364307    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-dshm" (OuterVolumeSpecName: "dshm") pod "63eee0a0-9016-434b-b4cc-9d6205bc4c5f" (UID: "63eee0a0-9016-434b-b4cc-9d6205bc4c5f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:40.364422 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.364335    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-kube-api-access-f7dgw" (OuterVolumeSpecName: "kube-api-access-f7dgw") pod "63eee0a0-9016-434b-b4cc-9d6205bc4c5f" (UID: "63eee0a0-9016-434b-b4cc-9d6205bc4c5f"). InnerVolumeSpecName "kube-api-access-f7dgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:40.463722 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.463691    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7dgw\" (UniqueName: \"kubernetes.io/projected/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-kube-api-access-f7dgw\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:56:40.463722 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.463720    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:56:40.463722 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.463731    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63eee0a0-9016-434b-b4cc-9d6205bc4c5f-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:56:40.916870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.916839    2550 generic.go:356] "Generic (PLEG): container finished" podID="63eee0a0-9016-434b-b4cc-9d6205bc4c5f" containerID="89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5" exitCode=0
Sep 09 20:56:40.917366 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.916883    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"
Sep 09 20:56:40.917366 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.916893    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6" event={"ID":"63eee0a0-9016-434b-b4cc-9d6205bc4c5f","Type":"ContainerDied","Data":"89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5"}
Sep 09 20:56:40.917366 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.916943    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6" event={"ID":"63eee0a0-9016-434b-b4cc-9d6205bc4c5f","Type":"ContainerDied","Data":"ac4c71b6c43d2714f7fe90adce25dc371e120d0c9034aa667325e83058e86ee4"}
Sep 09 20:56:40.917366 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.916966    2550 scope.go:117] "RemoveContainer" containerID="89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5"
Sep 09 20:56:40.927005 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.926988    2550 scope.go:117] "RemoveContainer" containerID="89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5"
Sep 09 20:56:40.927301 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:56:40.927282    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5\": container with ID starting with 89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5 not found: ID does not exist" containerID="89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5"
Sep 09 20:56:40.927360 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.927308    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5"} err="failed to get container status \"89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5\": rpc error: code = NotFound desc = could not find container \"89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5\": container with ID starting with 89b96db0492a88d04ff73ad679c8d81104d3cd73fbd953c59472f5beea3b0ec5 not found: ID does not exist"
Sep 09 20:56:40.940134 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.940108    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"]
Sep 09 20:56:40.947203 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:40.947174    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-fbe7be47/e2e-lifecycle-model-fbe7be47-kserve-ff54bc48c-tfhw6"]
Sep 09 20:56:42.288345 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:56:42.288314    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63eee0a0-9016-434b-b4cc-9d6205bc4c5f" path="/var/lib/kubelet/pods/63eee0a0-9016-434b-b4cc-9d6205bc4c5f/volumes"
Sep 09 20:57:01.236139 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:01.236096    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:02.474159 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:02.474119    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:08.173289 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:08.173252    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:09.338025 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:09.337982    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:10.738719 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:10.738670    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:14.168643 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:14.168613    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"]
Sep 09 20:57:14.169029 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:14.168807    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy" containerID="cri-o://3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b" gracePeriod=30
Sep 09 20:57:15.156058 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:15.156025    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:15.257673 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:15.257639    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:15.433557 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:15.433470    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:15.529632 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:15.529599    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:15.632320 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:15.632284    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:16.156523 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:16.156487    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:16.258183 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:16.258127    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:16.360012 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:16.359977    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:17.554289 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:17.554255    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:19.422246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.422219    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:57:19.555431 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555406    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-credential-socket\") pod \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") "
Sep 09 20:57:19.555588 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555469    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-envoy\") pod \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") "
Sep 09 20:57:19.555588 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555491    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-certs\") pod \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") "
Sep 09 20:57:19.555588 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555520    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-podinfo\") pod \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") "
Sep 09 20:57:19.555588 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555550    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-token\") pod \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") "
Sep 09 20:57:19.555588 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555579    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5d2n\" (UniqueName: \"kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-kube-api-access-p5d2n\") pod \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") "
Sep 09 20:57:19.555866 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555639    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-socket\") pod \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") "
Sep 09 20:57:19.555866 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555654    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a4b1801e-a3b5-430d-85f9-86a493ce8ca5" (UID: "a4b1801e-a3b5-430d-85f9-86a493ce8ca5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:19.555866 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555679    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istiod-ca-cert\") pod \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") "
Sep 09 20:57:19.555866 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555705    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-data\") pod \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\" (UID: \"a4b1801e-a3b5-430d-85f9-86a493ce8ca5\") "
Sep 09 20:57:19.555866 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555804    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a4b1801e-a3b5-430d-85f9-86a493ce8ca5" (UID: "a4b1801e-a3b5-430d-85f9-86a493ce8ca5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:19.556124 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555895    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a4b1801e-a3b5-430d-85f9-86a493ce8ca5" (UID: "a4b1801e-a3b5-430d-85f9-86a493ce8ca5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:19.556124 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.555985    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-data" (OuterVolumeSpecName: "istio-data") pod "a4b1801e-a3b5-430d-85f9-86a493ce8ca5" (UID: "a4b1801e-a3b5-430d-85f9-86a493ce8ca5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:19.556279 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.556127    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:57:19.556279 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.556144    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:57:19.556279 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.556158    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:57:19.556279 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.556195    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:57:19.556279 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.556211    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a4b1801e-a3b5-430d-85f9-86a493ce8ca5" (UID: "a4b1801e-a3b5-430d-85f9-86a493ce8ca5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:57:19.557845 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.557814    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-kube-api-access-p5d2n" (OuterVolumeSpecName: "kube-api-access-p5d2n") pod "a4b1801e-a3b5-430d-85f9-86a493ce8ca5" (UID: "a4b1801e-a3b5-430d-85f9-86a493ce8ca5"). InnerVolumeSpecName "kube-api-access-p5d2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:57:19.557845 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.557819    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a4b1801e-a3b5-430d-85f9-86a493ce8ca5" (UID: "a4b1801e-a3b5-430d-85f9-86a493ce8ca5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:19.557987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.557851    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4b1801e-a3b5-430d-85f9-86a493ce8ca5" (UID: "a4b1801e-a3b5-430d-85f9-86a493ce8ca5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:57:19.557987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.557848    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a4b1801e-a3b5-430d-85f9-86a493ce8ca5" (UID: "a4b1801e-a3b5-430d-85f9-86a493ce8ca5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:57:19.657328 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.657303    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:57:19.657328 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.657325    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:57:19.657470 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.657334    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5d2n\" (UniqueName: \"kubernetes.io/projected/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-kube-api-access-p5d2n\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:57:19.657470 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.657344    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:57:19.657470 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:19.657352    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4b1801e-a3b5-430d-85f9-86a493ce8ca5-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:57:20.161799 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.161765    2550 generic.go:356] "Generic (PLEG): container finished" podID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerID="3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b" exitCode=0
Sep 09 20:57:20.162000 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.161818    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"
Sep 09 20:57:20.162000 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.161826    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" event={"ID":"a4b1801e-a3b5-430d-85f9-86a493ce8ca5","Type":"ContainerDied","Data":"3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b"}
Sep 09 20:57:20.162000 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.161962    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn" event={"ID":"a4b1801e-a3b5-430d-85f9-86a493ce8ca5","Type":"ContainerDied","Data":"141db6a2a45aee3b8a8751519ab524cf8e563339f08f65b75e405d67754e5910"}
Sep 09 20:57:20.162000 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.161984    2550 scope.go:117] "RemoveContainer" containerID="3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b"
Sep 09 20:57:20.172100 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.172081    2550 scope.go:117] "RemoveContainer" containerID="cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1"
Sep 09 20:57:20.180869 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.180852    2550 scope.go:117] "RemoveContainer" containerID="3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b"
Sep 09 20:57:20.181159 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:57:20.181134    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b\": container with ID starting with 3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b not found: ID does not exist" containerID="3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b"
Sep 09 20:57:20.181277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.181185    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b"} err="failed to get container status \"3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b\": rpc error: code = NotFound desc = could not find container \"3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b\": container with ID starting with 3c290a549fd7f5f4fba4b4a382fb11e7f2b0b80a425b99fd980533a3adcd080b not found: ID does not exist"
Sep 09 20:57:20.181277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.181210    2550 scope.go:117] "RemoveContainer" containerID="cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1"
Sep 09 20:57:20.181531 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:57:20.181508    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1\": container with ID starting with cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1 not found: ID does not exist" containerID="cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1"
Sep 09 20:57:20.181655 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.181536    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1"} err="failed to get container status \"cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1\": rpc error: code = NotFound desc = could not find container \"cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1\": container with ID starting with cf1f2e64631ed5851d2f806647cfdbc118adb4e9402d418f396b01dc9409cfd1 not found: ID does not exist"
Sep 09 20:57:20.184246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.184226    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"]
Sep 09 20:57:20.189971 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.189952    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-fbe7be47-openshift-default-699b77ffc6-ltlsn"]
Sep 09 20:57:20.281033 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.281006    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" path="/var/lib/kubelet/pods/a4b1801e-a3b5-430d-85f9-86a493ce8ca5/volumes"
Sep 09 20:57:20.392826 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.392801    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:57:20.396676 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.396657    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 20:57:20.812106 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:20.812017    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:23.560303 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:23.560256    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:23.662683 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:23.662646    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:24.222425 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:24.222386    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:26.431186 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:26.431123    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:28.682464 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.682431    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"]
Sep 09 20:57:28.683029 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.683008    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy"
Sep 09 20:57:28.683117 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.683032    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy"
Sep 09 20:57:28.683117 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.683072    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63eee0a0-9016-434b-b4cc-9d6205bc4c5f" containerName="main"
Sep 09 20:57:28.683117 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.683082    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="63eee0a0-9016-434b-b4cc-9d6205bc4c5f" containerName="main"
Sep 09 20:57:28.683304 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.683214    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy"
Sep 09 20:57:28.683304 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.683228    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy"
Sep 09 20:57:28.683304 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.683239    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="63eee0a0-9016-434b-b4cc-9d6205bc4c5f" containerName="main"
Sep 09 20:57:28.686481 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.686461    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.688609 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.688589    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-933e9d\"/\"embedding-w4-933e9d-kserve-self-signed-certs\""
Sep 09 20:57:28.695913 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.695891    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"]
Sep 09 20:57:28.706711 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.706676    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:28.844771 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.844730    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-home\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.844978 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.844784    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-model-cache\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.844978 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.844852    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tls-certs\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.844978 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.844923    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzxp6\" (UniqueName: \"kubernetes.io/projected/6e579cc1-f249-4168-8f97-4f1c5e3c241a-kube-api-access-bzxp6\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.845148 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.844995    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-dshm\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.845148 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.845064    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tmp-dir\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.946489 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.946399    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tls-certs\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.946650 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.946568    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzxp6\" (UniqueName: \"kubernetes.io/projected/6e579cc1-f249-4168-8f97-4f1c5e3c241a-kube-api-access-bzxp6\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.946650 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.946629    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-dshm\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.946761 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.946676    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tmp-dir\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.946761 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.946714    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-home\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.946761 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.946747    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-model-cache\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.947084 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.947057    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tmp-dir\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.947084 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.947080    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-model-cache\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.947247 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.947154    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-home\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.948775 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.948753    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-dshm\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.949005 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.948988    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tls-certs\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.954082 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.954061    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzxp6\" (UniqueName: \"kubernetes.io/projected/6e579cc1-f249-4168-8f97-4f1c5e3c241a-kube-api-access-bzxp6\") pod \"embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") " pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:28.996217 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:28.996188    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:29.273921 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:29.273890    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"]
Sep 09 20:57:29.275526 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:57:29.275500    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e579cc1_f249_4168_8f97_4f1c5e3c241a.slice/crio-a4e462ca4221333f277baab11b4fa400c29e81bdb6c5a5888622c540c794a9b0 WatchSource:0}: Error finding container a4e462ca4221333f277baab11b4fa400c29e81bdb6c5a5888622c540c794a9b0: Status 404 returned error can't find the container with id a4e462ca4221333f277baab11b4fa400c29e81bdb6c5a5888622c540c794a9b0
Sep 09 20:57:30.039785 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:30.039726    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:30.149874 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:30.149840    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:30.222465 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:30.222424    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng" event={"ID":"6e579cc1-f249-4168-8f97-4f1c5e3c241a","Type":"ContainerStarted","Data":"d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8"}
Sep 09 20:57:30.222635 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:30.222472    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng" event={"ID":"6e579cc1-f249-4168-8f97-4f1c5e3c241a","Type":"ContainerStarted","Data":"a4e462ca4221333f277baab11b4fa400c29e81bdb6c5a5888622c540c794a9b0"}
Sep 09 20:57:30.242021 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:30.241973    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng" podStartSLOduration=2.241959145 podStartE2EDuration="2.241959145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:57:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:30.238943896 +0000 UTC m=+1510.389616240" watchObservedRunningTime="2026-09-09 20:57:30.241959145 +0000 UTC m=+1510.392631543"
Sep 09 20:57:30.243382 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:30.243352    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:37.402860 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.402831    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"]
Sep 09 20:57:37.403325 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.403289    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy"
Sep 09 20:57:37.403325 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.403301    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4b1801e-a3b5-430d-85f9-86a493ce8ca5" containerName="istio-proxy"
Sep 09 20:57:37.406778 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.406756    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.408878 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.408853    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2b2343\"/\"embedding-w0-2b2343-kserve-self-signed-certs\""
Sep 09 20:57:37.417910 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.417890    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"]
Sep 09 20:57:37.432021 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.431994    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:37.436208 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.435765    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-dshm\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.436327 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.436310    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-home\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.436395 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.436354    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-model-cache\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.436395 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.436386    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tls-certs\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.436502 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.436486    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbjgm\" (UniqueName: \"kubernetes.io/projected/694b282b-0d69-4778-b2af-dd47c6b3c6b0-kube-api-access-dbjgm\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.436555 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.436511    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tmp-dir\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.537982 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.537946    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-dshm\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.538197 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.537995    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-home\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.538197 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.538016    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-model-cache\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.538197 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.538042    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tls-certs\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.538197 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.538118    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbjgm\" (UniqueName: \"kubernetes.io/projected/694b282b-0d69-4778-b2af-dd47c6b3c6b0-kube-api-access-dbjgm\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.538197 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.538139    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tmp-dir\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.538521 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.538497    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-model-cache\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.538573 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.538528    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-home\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.538573 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.538547    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tmp-dir\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.540804 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.540779    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-dshm\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.541041 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.541025    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tls-certs\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.546985 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.546961    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbjgm\" (UniqueName: \"kubernetes.io/projected/694b282b-0d69-4778-b2af-dd47c6b3c6b0-kube-api-access-dbjgm\") pod \"embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") " pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.716970 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.716923    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:37.991542 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:37.991511    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"]
Sep 09 20:57:37.992484 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:57:37.992461    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod694b282b_0d69_4778_b2af_dd47c6b3c6b0.slice/crio-4fb406e0c4becd3ac3b0383ecd224448a7dc886b7fc17824c2f581cf3fce1145 WatchSource:0}: Error finding container 4fb406e0c4becd3ac3b0383ecd224448a7dc886b7fc17824c2f581cf3fce1145: Status 404 returned error can't find the container with id 4fb406e0c4becd3ac3b0383ecd224448a7dc886b7fc17824c2f581cf3fce1145
Sep 09 20:57:38.272277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:38.272198    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg" event={"ID":"694b282b-0d69-4778-b2af-dd47c6b3c6b0","Type":"ContainerStarted","Data":"b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e"}
Sep 09 20:57:38.272277 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:38.272240    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg" event={"ID":"694b282b-0d69-4778-b2af-dd47c6b3c6b0","Type":"ContainerStarted","Data":"4fb406e0c4becd3ac3b0383ecd224448a7dc886b7fc17824c2f581cf3fce1145"}
Sep 09 20:57:38.290284 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:38.290239    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg" podStartSLOduration=1.290216896 podStartE2EDuration="1.290216896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:38.288795103 +0000 UTC m=+1518.439467446" watchObservedRunningTime="2026-09-09 20:57:38.290216896 +0000 UTC m=+1518.440889239"
Sep 09 20:57:38.996819 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:38.996788    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:38.997193 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:38.997059    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:39.009035 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:39.009006    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:39.288493 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:39.288430    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 20:57:40.673644 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:40.673612    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:40.922281 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:40.922246    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:47.718159 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:47.718075    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:47.718159 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:47.718121    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:47.730623 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:47.730594    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:48.341663 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:48.341630    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:57:54.760604 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:54.760559    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:55.150914 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:57:55.150793    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:01.761494 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:01.761455    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:07.721475 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:07.721429    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:07.805464 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:07.805434    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:08.053461 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:08.053378    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:08.387394 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:08.387305    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:08.544539 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:08.544499    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:11.676134 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:11.676100    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:11.776407 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:11.776374    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:13.007396 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:13.007363    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:13.105039 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:13.105002    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:13.634796 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:13.634761    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:13.735015 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:13.734977    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:35.495460 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:35.495420    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:35.560590 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:35.560559    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:36.616632 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:36.616596    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:37.031433 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.031387    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:37.309298 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.309202    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"]
Sep 09 20:58:37.313181 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.313149    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.319281 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.319257    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-4e2f93-openshift-default-dockercfg-4qj6x\""
Sep 09 20:58:37.348813 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.348785    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"]
Sep 09 20:58:37.369399 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.369363    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:37.445809 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.445771    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-credential-socket\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.445987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.445833    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-data\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.445987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.445913    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-certs\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.445987 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.445978    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krvd9\" (UniqueName: \"kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-kube-api-access-krvd9\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.446096 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.446020    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-envoy\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.446129 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.446098    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-token\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.446188 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.446138    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae1828bc-806a-4e5b-8320-22fd7df34953-istiod-ca-cert\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.446231 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.446160    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-socket\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.446289 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.446276    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-podinfo\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.547711 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.547678    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae1828bc-806a-4e5b-8320-22fd7df34953-istiod-ca-cert\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.547711 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.547718    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-socket\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.547903 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.547798    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-podinfo\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.547903 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.547824    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-credential-socket\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.547903 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.547875    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-data\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.548044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.547914    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-certs\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.548044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.547945    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krvd9\" (UniqueName: \"kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-kube-api-access-krvd9\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.548044 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.547974    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-envoy\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.548219 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.548045    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-token\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.548292 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.548255    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-socket\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.548352 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.548292    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-credential-socket\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.548407 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.548349    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-data\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.548407 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.548385    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-certs\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.548511 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.548494    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae1828bc-806a-4e5b-8320-22fd7df34953-istiod-ca-cert\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.550237 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.550213    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-envoy\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.550342 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.550257    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-podinfo\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.560860 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.560809    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krvd9\" (UniqueName: \"kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-kube-api-access-krvd9\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.582394 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.582366    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-token\") pod \"e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") " pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.622415 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.622387    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:37.623328 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.623311    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:37.905071 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:58:37.905037    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1828bc_806a_4e5b_8320_22fd7df34953.slice/crio-670a7324aaa233bab4f48e48bfb72d4d38278add4e0cac5afc34e49b1f01afd8 WatchSource:0}: Error finding container 670a7324aaa233bab4f48e48bfb72d4d38278add4e0cac5afc34e49b1f01afd8: Status 404 returned error can't find the container with id 670a7324aaa233bab4f48e48bfb72d4d38278add4e0cac5afc34e49b1f01afd8
Sep 09 20:58:37.905792 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.905769    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"]
Sep 09 20:58:37.907378 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.907343    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:58:37.907465 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.907408    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:58:37.907465 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:37.907437    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:58:38.666983 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:38.666953    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" event={"ID":"ae1828bc-806a-4e5b-8320-22fd7df34953","Type":"ContainerStarted","Data":"8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91"}
Sep 09 20:58:38.667370 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:38.667000    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" event={"ID":"ae1828bc-806a-4e5b-8320-22fd7df34953","Type":"ContainerStarted","Data":"670a7324aaa233bab4f48e48bfb72d4d38278add4e0cac5afc34e49b1f01afd8"}
Sep 09 20:58:38.689304 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:38.689256    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" podStartSLOduration=1.689239729 podStartE2EDuration="1.689239729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:38.686423846 +0000 UTC m=+1578.837096190" watchObservedRunningTime="2026-09-09 20:58:38.689239729 +0000 UTC m=+1578.839912071"
Sep 09 20:58:39.624283 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:39.624242    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:39.630490 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:39.630462    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:39.673977 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:39.673947    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:39.674911 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:39.674893    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:58:40.579840 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:40.579795    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:40.669687 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:40.669658    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:41.772436 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:41.772405    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:41.874032 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:41.873999    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:42.371154 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.371121    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:42.471952 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.471917    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:42.562049 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.562018    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"]
Sep 09 20:58:42.565771 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.565754    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:58:42.568415 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.568394    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-4e2f93\""
Sep 09 20:58:42.568530 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.568467    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-4e2f93-dockercfg-tx74z\""
Sep 09 20:58:42.575525 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.575502    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"]
Sep 09 20:58:42.701956 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.701878    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6422b306-5913-4321-bdd5-e40ef4a961fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8\" (UID: \"6422b306-5913-4321-bdd5-e40ef4a961fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:58:42.702101 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.701953    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlxct\" (UniqueName: \"kubernetes.io/projected/6422b306-5913-4321-bdd5-e40ef4a961fa-kube-api-access-qlxct\") pod \"payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8\" (UID: \"6422b306-5913-4321-bdd5-e40ef4a961fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:58:42.704850 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.704815    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:42.803665 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.803626    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6422b306-5913-4321-bdd5-e40ef4a961fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8\" (UID: \"6422b306-5913-4321-bdd5-e40ef4a961fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:58:42.804055 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.803686    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlxct\" (UniqueName: \"kubernetes.io/projected/6422b306-5913-4321-bdd5-e40ef4a961fa-kube-api-access-qlxct\") pod \"payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8\" (UID: \"6422b306-5913-4321-bdd5-e40ef4a961fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:58:42.804419 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.804400    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6422b306-5913-4321-bdd5-e40ef4a961fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8\" (UID: \"6422b306-5913-4321-bdd5-e40ef4a961fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:58:42.813056 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.813022    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlxct\" (UniqueName: \"kubernetes.io/projected/6422b306-5913-4321-bdd5-e40ef4a961fa-kube-api-access-qlxct\") pod \"payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8\" (UID: \"6422b306-5913-4321-bdd5-e40ef4a961fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:58:42.876975 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:42.876943    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:58:43.162840 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:43.162815    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"]
Sep 09 20:58:43.164504 ip-10-0-140-126 kubenswrapper[2550]: W0909 20:58:43.164474    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6422b306_5913_4321_bdd5_e40ef4a961fa.slice/crio-89391f395cc02a524f292b685d787abfcb68713b0d273d7b88430e05ff21a9ba WatchSource:0}: Error finding container 89391f395cc02a524f292b685d787abfcb68713b0d273d7b88430e05ff21a9ba: Status 404 returned error can't find the container with id 89391f395cc02a524f292b685d787abfcb68713b0d273d7b88430e05ff21a9ba
Sep 09 20:58:43.698322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:43.698196    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8" event={"ID":"6422b306-5913-4321-bdd5-e40ef4a961fa","Type":"ContainerStarted","Data":"785e17b496f745837f48b926ef1cab1a26a8e038dd27d6240e2f93f529669626"}
Sep 09 20:58:43.698322 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:43.698258    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8" event={"ID":"6422b306-5913-4321-bdd5-e40ef4a961fa","Type":"ContainerStarted","Data":"89391f395cc02a524f292b685d787abfcb68713b0d273d7b88430e05ff21a9ba"}
Sep 09 20:58:43.711781 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:43.698328    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:58:43.718898 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:43.718833    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8" podStartSLOduration=1.718818191 podStartE2EDuration="1.718818191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:43.717506827 +0000 UTC m=+1583.868179171" watchObservedRunningTime="2026-09-09 20:58:43.718818191 +0000 UTC m=+1583.869490572"
Sep 09 20:58:52.915230 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:52.915194    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:53.015332 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:53.015294    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:53.760558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:53.760525    2550 generic.go:356] "Generic (PLEG): container finished" podID="3d05d316-d3e3-40bb-888f-7d821ca1d2f4" containerID="485b0880caf3a286bda627eaf289ebb4b8bafdd8e3934899bd7c8ac2bb6737ec" exitCode=137
Sep 09 20:58:53.760759 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:53.760602    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r" event={"ID":"3d05d316-d3e3-40bb-888f-7d821ca1d2f4","Type":"ContainerDied","Data":"485b0880caf3a286bda627eaf289ebb4b8bafdd8e3934899bd7c8ac2bb6737ec"}
Sep 09 20:58:53.761006 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:53.760992    2550 scope.go:117] "RemoveContainer" containerID="485b0880caf3a286bda627eaf289ebb4b8bafdd8e3934899bd7c8ac2bb6737ec"
Sep 09 20:58:54.707153 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:54.707125    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:58:54.769392 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:54.769359    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r" event={"ID":"3d05d316-d3e3-40bb-888f-7d821ca1d2f4","Type":"ContainerStarted","Data":"45ae96896dd944bbd57a44b9811dcb716f7d627bb7acaa09bee4784a738f0bd9"}
Sep 09 20:58:54.769591 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:58:54.769573    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:59:05.486810 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:05.486773    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:05.778541 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:05.778460    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 20:59:08.359849 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:08.359820    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:11.319248 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:11.319211    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:23.943870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:23.943812    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:24.045471 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:24.045432    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:24.144441 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:24.144404    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:24.202523 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:24.202440    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:24.498580 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:24.498544    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:30.347184 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:30.347134    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:30.407186 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:30.407139    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:30.610607 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:30.610512    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:31.128858 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:31.128820    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:38.621204 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:38.621141    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:39.675882 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:39.675847    2550 patch_prober.go:28] interesting pod/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 09 20:59:39.676318 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:39.675931    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 09 20:59:39.894744 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:39.894662    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:40.064764 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:40.064736    2550 generic.go:356] "Generic (PLEG): container finished" podID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerID="8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91" exitCode=0
Sep 09 20:59:40.064931 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:40.064813    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" event={"ID":"ae1828bc-806a-4e5b-8320-22fd7df34953","Type":"ContainerDied","Data":"8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91"}
Sep 09 20:59:40.065243 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:40.065230    2550 scope.go:117] "RemoveContainer" containerID="8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91"
Sep 09 20:59:40.066523 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:40.066493    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:59:40.066602 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:40.066566    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:59:40.066645 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:40.066604    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:59:41.072926 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:41.072892    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" event={"ID":"ae1828bc-806a-4e5b-8320-22fd7df34953","Type":"ContainerStarted","Data":"0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8"}
Sep 09 20:59:41.138989 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:41.138946    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:41.624485 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:41.624447    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:59:41.624662 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:41.624613    2550 patch_prober.go:28] interesting pod/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 09 20:59:41.624708 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:41.624663    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 09 20:59:42.623746 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:42.623699    2550 patch_prober.go:28] interesting pod/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 09 20:59:42.624112 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:42.623769    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 09 20:59:43.619763 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:43.619731    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"]
Sep 09 20:59:43.624040 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:43.624007    2550 patch_prober.go:28] interesting pod/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 09 20:59:43.624423 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:43.624057    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 09 20:59:43.635189 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:43.631511    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"]
Sep 09 20:59:43.651048 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:43.651019    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"]
Sep 09 20:59:43.722811 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:43.722772    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:44.624487 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:44.624444    2550 patch_prober.go:28] interesting pod/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 09 20:59:44.624846 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:44.624510    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 09 20:59:45.623954 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:45.623914    2550 patch_prober.go:28] interesting pod/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 09 20:59:45.624154 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:45.623975    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 09 20:59:46.628348 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:46.628317    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:59:46.628701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:46.628573    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:59:46.629232 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:46.629217    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:59:46.822419 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:46.822387    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:46.861338 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:46.861304    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"]
Sep 09 20:59:46.861632 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:46.861547    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8" podUID="6422b306-5913-4321-bdd5-e40ef4a961fa" containerName="payload-pre-processing" containerID="cri-o://785e17b496f745837f48b926ef1cab1a26a8e038dd27d6240e2f93f529669626" gracePeriod=30
Sep 09 20:59:46.924749 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:46.924717    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:47.028793 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.028734    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:47.110649 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.110623    2550 generic.go:356] "Generic (PLEG): container finished" podID="6422b306-5913-4321-bdd5-e40ef4a961fa" containerID="785e17b496f745837f48b926ef1cab1a26a8e038dd27d6240e2f93f529669626" exitCode=0
Sep 09 20:59:47.110776 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.110661    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8" event={"ID":"6422b306-5913-4321-bdd5-e40ef4a961fa","Type":"ContainerDied","Data":"785e17b496f745837f48b926ef1cab1a26a8e038dd27d6240e2f93f529669626"}
Sep 09 20:59:47.110776 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.110713    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8" event={"ID":"6422b306-5913-4321-bdd5-e40ef4a961fa","Type":"ContainerDied","Data":"89391f395cc02a524f292b685d787abfcb68713b0d273d7b88430e05ff21a9ba"}
Sep 09 20:59:47.110776 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.110734    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89391f395cc02a524f292b685d787abfcb68713b0d273d7b88430e05ff21a9ba"
Sep 09 20:59:47.111351 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.111334    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:59:47.123411 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.123380    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:47.307249 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.307214    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6422b306-5913-4321-bdd5-e40ef4a961fa-plugins-config-volume\") pod \"6422b306-5913-4321-bdd5-e40ef4a961fa\" (UID: \"6422b306-5913-4321-bdd5-e40ef4a961fa\") "
Sep 09 20:59:47.307513 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.307490    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlxct\" (UniqueName: \"kubernetes.io/projected/6422b306-5913-4321-bdd5-e40ef4a961fa-kube-api-access-qlxct\") pod \"6422b306-5913-4321-bdd5-e40ef4a961fa\" (UID: \"6422b306-5913-4321-bdd5-e40ef4a961fa\") "
Sep 09 20:59:47.307603 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.307572    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6422b306-5913-4321-bdd5-e40ef4a961fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6422b306-5913-4321-bdd5-e40ef4a961fa" (UID: "6422b306-5913-4321-bdd5-e40ef4a961fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:47.308120 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.308078    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6422b306-5913-4321-bdd5-e40ef4a961fa-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:47.309596 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.309577    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6422b306-5913-4321-bdd5-e40ef4a961fa-kube-api-access-qlxct" (OuterVolumeSpecName: "kube-api-access-qlxct") pod "6422b306-5913-4321-bdd5-e40ef4a961fa" (UID: "6422b306-5913-4321-bdd5-e40ef4a961fa"). InnerVolumeSpecName "kube-api-access-qlxct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:47.397543 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.397499    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:47.408921 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:47.408892    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlxct\" (UniqueName: \"kubernetes.io/projected/6422b306-5913-4321-bdd5-e40ef4a961fa-kube-api-access-qlxct\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:48.117571 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:48.117533    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"
Sep 09 20:59:48.139104 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:48.139077    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"]
Sep 09 20:59:48.141997 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:48.141974    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2f93-5c75cf57cd-bfjr8"]
Sep 09 20:59:48.280040 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:48.280013    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6422b306-5913-4321-bdd5-e40ef4a961fa" path="/var/lib/kubelet/pods/6422b306-5913-4321-bdd5-e40ef4a961fa/volumes"
Sep 09 20:59:49.090246 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:49.090213    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:49.844340 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:49.844305    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:53.448723 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:53.448636    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:53.550836 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:53.550798    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:53.682696 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:53.682656    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"]
Sep 09 20:59:53.682893 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:53.682868    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy" containerID="cri-o://0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8" gracePeriod=30
Sep 09 20:59:53.711095 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:53.711067    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:54.247913 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:54.247868    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:55.499102 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:55.499059    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:55.571734 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:55.571701    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:55.645798 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:55.645675    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:55.729649 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:55.729618    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:55.858442 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:55.858350    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:56.085392 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:56.085363    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:58.668159 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:58.668125    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg" podUID="694b282b-0d69-4778-b2af-dd47c6b3c6b0" containerName="main" containerID="cri-o://b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e" gracePeriod=45
Sep 09 20:59:58.682813 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:58.682755    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4" podUID="5e45b1a2-815c-476d-980e-c505c810c987" containerName="main" containerID="cri-o://da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154" gracePeriod=45
Sep 09 20:59:58.682813 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:58.682786    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn" podUID="a5e1b010-99bb-42de-94de-618e9cd83e91" containerName="main" containerID="cri-o://59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0" gracePeriod=45
Sep 09 20:59:58.991996 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:58.991670    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:59:59.018685 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.018340    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-model-cache\") pod \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") "
Sep 09 20:59:59.018685 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.018404    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tmp-dir\") pod \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") "
Sep 09 20:59:59.018685 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.018460    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbjgm\" (UniqueName: \"kubernetes.io/projected/694b282b-0d69-4778-b2af-dd47c6b3c6b0-kube-api-access-dbjgm\") pod \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") "
Sep 09 20:59:59.018685 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.018499    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-dshm\") pod \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") "
Sep 09 20:59:59.018685 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.018527    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-home\") pod \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") "
Sep 09 20:59:59.018685 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.018551    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tls-certs\") pod \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\" (UID: \"694b282b-0d69-4778-b2af-dd47c6b3c6b0\") "
Sep 09 20:59:59.019950 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.019870    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-model-cache" (OuterVolumeSpecName: "model-cache") pod "694b282b-0d69-4778-b2af-dd47c6b3c6b0" (UID: "694b282b-0d69-4778-b2af-dd47c6b3c6b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.020654 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.020612    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "694b282b-0d69-4778-b2af-dd47c6b3c6b0" (UID: "694b282b-0d69-4778-b2af-dd47c6b3c6b0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.021127 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.021095    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-home" (OuterVolumeSpecName: "home") pod "694b282b-0d69-4778-b2af-dd47c6b3c6b0" (UID: "694b282b-0d69-4778-b2af-dd47c6b3c6b0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.023337 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.023254    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/694b282b-0d69-4778-b2af-dd47c6b3c6b0-kube-api-access-dbjgm" (OuterVolumeSpecName: "kube-api-access-dbjgm") pod "694b282b-0d69-4778-b2af-dd47c6b3c6b0" (UID: "694b282b-0d69-4778-b2af-dd47c6b3c6b0"). InnerVolumeSpecName "kube-api-access-dbjgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:59.024470 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.024446    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-dshm" (OuterVolumeSpecName: "dshm") pod "694b282b-0d69-4778-b2af-dd47c6b3c6b0" (UID: "694b282b-0d69-4778-b2af-dd47c6b3c6b0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.024470 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.024453    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "694b282b-0d69-4778-b2af-dd47c6b3c6b0" (UID: "694b282b-0d69-4778-b2af-dd47c6b3c6b0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:59:59.030038 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.030009    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:59:59.041354 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.041314    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:59:59.043570 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.043548    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:59:59.119884 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.119855    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-socket\") pod \"ae1828bc-806a-4e5b-8320-22fd7df34953\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") "
Sep 09 20:59:59.120070 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.119894    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-token\") pod \"ae1828bc-806a-4e5b-8320-22fd7df34953\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") "
Sep 09 20:59:59.120070 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.119918    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e45b1a2-815c-476d-980e-c505c810c987-tls-certs\") pod \"5e45b1a2-815c-476d-980e-c505c810c987\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") "
Sep 09 20:59:59.120070 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.119942    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-home\") pod \"5e45b1a2-815c-476d-980e-c505c810c987\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") "
Sep 09 20:59:59.120070 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120000    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-tmp-dir\") pod \"a5e1b010-99bb-42de-94de-618e9cd83e91\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") "
Sep 09 20:59:59.120070 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120032    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae1828bc-806a-4e5b-8320-22fd7df34953-istiod-ca-cert\") pod \"ae1828bc-806a-4e5b-8320-22fd7df34953\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") "
Sep 09 20:59:59.120369 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120083    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xz4d2\" (UniqueName: \"kubernetes.io/projected/5e45b1a2-815c-476d-980e-c505c810c987-kube-api-access-xz4d2\") pod \"5e45b1a2-815c-476d-980e-c505c810c987\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") "
Sep 09 20:59:59.120369 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120117    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-dshm\") pod \"5e45b1a2-815c-476d-980e-c505c810c987\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") "
Sep 09 20:59:59.120369 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120231    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae1828bc-806a-4e5b-8320-22fd7df34953" (UID: "ae1828bc-806a-4e5b-8320-22fd7df34953"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.120369 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120243    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-home" (OuterVolumeSpecName: "home") pod "5e45b1a2-815c-476d-980e-c505c810c987" (UID: "5e45b1a2-815c-476d-980e-c505c810c987"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.120369 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120317    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-certs\") pod \"ae1828bc-806a-4e5b-8320-22fd7df34953\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") "
Sep 09 20:59:59.120369 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120337    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a5e1b010-99bb-42de-94de-618e9cd83e91" (UID: "a5e1b010-99bb-42de-94de-618e9cd83e91"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120382    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjm77\" (UniqueName: \"kubernetes.io/projected/a5e1b010-99bb-42de-94de-618e9cd83e91-kube-api-access-pjm77\") pod \"a5e1b010-99bb-42de-94de-618e9cd83e91\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") "
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120410    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krvd9\" (UniqueName: \"kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-kube-api-access-krvd9\") pod \"ae1828bc-806a-4e5b-8320-22fd7df34953\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") "
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120440    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-tmp-dir\") pod \"5e45b1a2-815c-476d-980e-c505c810c987\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") "
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120442    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae1828bc-806a-4e5b-8320-22fd7df34953-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae1828bc-806a-4e5b-8320-22fd7df34953" (UID: "ae1828bc-806a-4e5b-8320-22fd7df34953"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120464    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-home\") pod \"a5e1b010-99bb-42de-94de-618e9cd83e91\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") "
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120494    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-data\") pod \"ae1828bc-806a-4e5b-8320-22fd7df34953\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") "
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120555    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-dshm\") pod \"a5e1b010-99bb-42de-94de-618e9cd83e91\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") "
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120583    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-envoy\") pod \"ae1828bc-806a-4e5b-8320-22fd7df34953\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") "
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120595    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae1828bc-806a-4e5b-8320-22fd7df34953" (UID: "ae1828bc-806a-4e5b-8320-22fd7df34953"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120622    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-credential-socket\") pod \"ae1828bc-806a-4e5b-8320-22fd7df34953\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") "
Sep 09 20:59:59.120677 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120675    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-model-cache\") pod \"5e45b1a2-815c-476d-980e-c505c810c987\" (UID: \"5e45b1a2-815c-476d-980e-c505c810c987\") "
Sep 09 20:59:59.121214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120700    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-model-cache\") pod \"a5e1b010-99bb-42de-94de-618e9cd83e91\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") "
Sep 09 20:59:59.121214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120733    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-podinfo\") pod \"ae1828bc-806a-4e5b-8320-22fd7df34953\" (UID: \"ae1828bc-806a-4e5b-8320-22fd7df34953\") "
Sep 09 20:59:59.121214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.120768    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5e1b010-99bb-42de-94de-618e9cd83e91-tls-certs\") pod \"a5e1b010-99bb-42de-94de-618e9cd83e91\" (UID: \"a5e1b010-99bb-42de-94de-618e9cd83e91\") "
Sep 09 20:59:59.121214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121185    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.121214 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121204    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.121477 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121217    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbjgm\" (UniqueName: \"kubernetes.io/projected/694b282b-0d69-4778-b2af-dd47c6b3c6b0-kube-api-access-dbjgm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.121477 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121229    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.121477 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121242    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/694b282b-0d69-4778-b2af-dd47c6b3c6b0-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.121477 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121254    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/694b282b-0d69-4778-b2af-dd47c6b3c6b0-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.121477 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121267    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.121477 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121280    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.121788 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121292    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.121788 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121575    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae1828bc-806a-4e5b-8320-22fd7df34953-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.121788 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.121588    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.122701 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.122632    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae1828bc-806a-4e5b-8320-22fd7df34953" (UID: "ae1828bc-806a-4e5b-8320-22fd7df34953"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:59.122870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.122818    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5e45b1a2-815c-476d-980e-c505c810c987" (UID: "5e45b1a2-815c-476d-980e-c505c810c987"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.122870 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.122822    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-home" (OuterVolumeSpecName: "home") pod "a5e1b010-99bb-42de-94de-618e9cd83e91" (UID: "a5e1b010-99bb-42de-94de-618e9cd83e91"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.123121 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.123011    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae1828bc-806a-4e5b-8320-22fd7df34953" (UID: "ae1828bc-806a-4e5b-8320-22fd7df34953"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.123121 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.123019    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-model-cache" (OuterVolumeSpecName: "model-cache") pod "a5e1b010-99bb-42de-94de-618e9cd83e91" (UID: "a5e1b010-99bb-42de-94de-618e9cd83e91"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.123326 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.123176    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e45b1a2-815c-476d-980e-c505c810c987" (UID: "5e45b1a2-815c-476d-980e-c505c810c987"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.123564 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.123528    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae1828bc-806a-4e5b-8320-22fd7df34953" (UID: "ae1828bc-806a-4e5b-8320-22fd7df34953"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.123726 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.123699    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e45b1a2-815c-476d-980e-c505c810c987-kube-api-access-xz4d2" (OuterVolumeSpecName: "kube-api-access-xz4d2") pod "5e45b1a2-815c-476d-980e-c505c810c987" (UID: "5e45b1a2-815c-476d-980e-c505c810c987"). InnerVolumeSpecName "kube-api-access-xz4d2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:59.123902 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.123872    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-dshm" (OuterVolumeSpecName: "dshm") pod "5e45b1a2-815c-476d-980e-c505c810c987" (UID: "5e45b1a2-815c-476d-980e-c505c810c987"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.124494 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.124449    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5e1b010-99bb-42de-94de-618e9cd83e91-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a5e1b010-99bb-42de-94de-618e9cd83e91" (UID: "a5e1b010-99bb-42de-94de-618e9cd83e91"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:59:59.124603 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.124578    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-kube-api-access-krvd9" (OuterVolumeSpecName: "kube-api-access-krvd9") pod "ae1828bc-806a-4e5b-8320-22fd7df34953" (UID: "ae1828bc-806a-4e5b-8320-22fd7df34953"). InnerVolumeSpecName "kube-api-access-krvd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:59.124747 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.124722    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae1828bc-806a-4e5b-8320-22fd7df34953" (UID: "ae1828bc-806a-4e5b-8320-22fd7df34953"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.124845 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.124795    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e45b1a2-815c-476d-980e-c505c810c987-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e45b1a2-815c-476d-980e-c505c810c987" (UID: "5e45b1a2-815c-476d-980e-c505c810c987"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:59:59.125483 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.125454    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5e1b010-99bb-42de-94de-618e9cd83e91-kube-api-access-pjm77" (OuterVolumeSpecName: "kube-api-access-pjm77") pod "a5e1b010-99bb-42de-94de-618e9cd83e91" (UID: "a5e1b010-99bb-42de-94de-618e9cd83e91"). InnerVolumeSpecName "kube-api-access-pjm77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:59.125893 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.125869    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-dshm" (OuterVolumeSpecName: "dshm") pod "a5e1b010-99bb-42de-94de-618e9cd83e91" (UID: "a5e1b010-99bb-42de-94de-618e9cd83e91"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:59.126123 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.126096    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae1828bc-806a-4e5b-8320-22fd7df34953" (UID: "ae1828bc-806a-4e5b-8320-22fd7df34953"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:59:59.205461 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.205427    2550 generic.go:356] "Generic (PLEG): container finished" podID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerID="0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8" exitCode=0
Sep 09 20:59:59.205642 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.205497    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"
Sep 09 20:59:59.205642 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.205514    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" event={"ID":"ae1828bc-806a-4e5b-8320-22fd7df34953","Type":"ContainerDied","Data":"0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8"}
Sep 09 20:59:59.205642 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.205605    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p" event={"ID":"ae1828bc-806a-4e5b-8320-22fd7df34953","Type":"ContainerDied","Data":"670a7324aaa233bab4f48e48bfb72d4d38278add4e0cac5afc34e49b1f01afd8"}
Sep 09 20:59:59.205642 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.205631    2550 scope.go:117] "RemoveContainer" containerID="0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8"
Sep 09 20:59:59.207058 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.207039    2550 generic.go:356] "Generic (PLEG): container finished" podID="5e45b1a2-815c-476d-980e-c505c810c987" containerID="da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154" exitCode=0
Sep 09 20:59:59.207203 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.207096    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"
Sep 09 20:59:59.207203 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.207115    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4" event={"ID":"5e45b1a2-815c-476d-980e-c505c810c987","Type":"ContainerDied","Data":"da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154"}
Sep 09 20:59:59.207203 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.207148    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4" event={"ID":"5e45b1a2-815c-476d-980e-c505c810c987","Type":"ContainerDied","Data":"301f8d6283e929637ac41096f767901ad4e26026a8f54ac9a2b208aa4d1cce8e"}
Sep 09 20:59:59.208593 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.208568    2550 generic.go:356] "Generic (PLEG): container finished" podID="a5e1b010-99bb-42de-94de-618e9cd83e91" containerID="59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0" exitCode=0
Sep 09 20:59:59.208712 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.208630    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"
Sep 09 20:59:59.208930 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.208654    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn" event={"ID":"a5e1b010-99bb-42de-94de-618e9cd83e91","Type":"ContainerDied","Data":"59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0"}
Sep 09 20:59:59.209098 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.209072    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn" event={"ID":"a5e1b010-99bb-42de-94de-618e9cd83e91","Type":"ContainerDied","Data":"bfe98055247b88eaab5bdc12ee8a09bbbd9453e9c4dcd8e3ad927af6017c3998"}
Sep 09 20:59:59.210404 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.210345    2550 generic.go:356] "Generic (PLEG): container finished" podID="694b282b-0d69-4778-b2af-dd47c6b3c6b0" containerID="b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e" exitCode=0
Sep 09 20:59:59.210404 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.210395    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"
Sep 09 20:59:59.210558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.210394    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg" event={"ID":"694b282b-0d69-4778-b2af-dd47c6b3c6b0","Type":"ContainerDied","Data":"b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e"}
Sep 09 20:59:59.210558 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.210507    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg" event={"ID":"694b282b-0d69-4778-b2af-dd47c6b3c6b0","Type":"ContainerDied","Data":"4fb406e0c4becd3ac3b0383ecd224448a7dc886b7fc17824c2f581cf3fce1145"}
Sep 09 20:59:59.219228 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.219209    2550 scope.go:117] "RemoveContainer" containerID="8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91"
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222078    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222097    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222107    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222115    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222122    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5e1b010-99bb-42de-94de-618e9cd83e91-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222130    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222137    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e45b1a2-815c-476d-980e-c505c810c987-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222145    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xz4d2\" (UniqueName: \"kubernetes.io/projected/5e45b1a2-815c-476d-980e-c505c810c987-kube-api-access-xz4d2\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222157    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222199    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjm77\" (UniqueName: \"kubernetes.io/projected/a5e1b010-99bb-42de-94de-618e9cd83e91-kube-api-access-pjm77\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222212    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krvd9\" (UniqueName: \"kubernetes.io/projected/ae1828bc-806a-4e5b-8320-22fd7df34953-kube-api-access-krvd9\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222240    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e45b1a2-815c-476d-980e-c505c810c987-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222249    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222257    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222265    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5e1b010-99bb-42de-94de-618e9cd83e91-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.222313 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.222284    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae1828bc-806a-4e5b-8320-22fd7df34953-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 20:59:59.229617 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.229598    2550 scope.go:117] "RemoveContainer" containerID="0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8"
Sep 09 20:59:59.229898 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:59:59.229874    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8\": container with ID starting with 0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8 not found: ID does not exist" containerID="0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8"
Sep 09 20:59:59.229960 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.229906    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8"} err="failed to get container status \"0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8\": rpc error: code = NotFound desc = could not find container \"0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8\": container with ID starting with 0726e4650d9cec5c45e1bed2c7d34d619bdac74bf76d663da5ddaa2018e2f6d8 not found: ID does not exist"
Sep 09 20:59:59.229960 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.229924    2550 scope.go:117] "RemoveContainer" containerID="8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91"
Sep 09 20:59:59.230202 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:59:59.230151    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91\": container with ID starting with 8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91 not found: ID does not exist" containerID="8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91"
Sep 09 20:59:59.230286 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.230205    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91"} err="failed to get container status \"8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91\": rpc error: code = NotFound desc = could not find container \"8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91\": container with ID starting with 8789d6d81ae16ed215f03cee14c2a4c728a386c3786086ed6ef103dde1dfec91 not found: ID does not exist"
Sep 09 20:59:59.230286 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.230223    2550 scope.go:117] "RemoveContainer" containerID="da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154"
Sep 09 20:59:59.237838 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.237812    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"]
Sep 09 20:59:59.241059 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.241043    2550 scope.go:117] "RemoveContainer" containerID="da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154"
Sep 09 20:59:59.241408 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:59:59.241389    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154\": container with ID starting with da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154 not found: ID does not exist" containerID="da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154"
Sep 09 20:59:59.241480 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.241413    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154"} err="failed to get container status \"da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154\": rpc error: code = NotFound desc = could not find container \"da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154\": container with ID starting with da2ee39a539e81e93ea967a89d5e809739b1f5ab856c01cff48105e81d008154 not found: ID does not exist"
Sep 09 20:59:59.241480 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.241430    2550 scope.go:117] "RemoveContainer" containerID="59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0"
Sep 09 20:59:59.242152 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.242134    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4e2f93-openshift-default-6c89b97f84-l6v4p"]
Sep 09 20:59:59.249271 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.249255    2550 scope.go:117] "RemoveContainer" containerID="59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0"
Sep 09 20:59:59.249510 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:59:59.249493    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0\": container with ID starting with 59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0 not found: ID does not exist" containerID="59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0"
Sep 09 20:59:59.249555 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.249517    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0"} err="failed to get container status \"59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0\": rpc error: code = NotFound desc = could not find container \"59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0\": container with ID starting with 59c64f9843dbd96986ca8ca4a136473852c9be1055007375527da30fa7fdbbb0 not found: ID does not exist"
Sep 09 20:59:59.249555 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.249532    2550 scope.go:117] "RemoveContainer" containerID="b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e"
Sep 09 20:59:59.253981 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.253959    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"]
Sep 09 20:59:59.257973 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.257957    2550 scope.go:117] "RemoveContainer" containerID="b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e"
Sep 09 20:59:59.258135 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.258115    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/premium-w0-2b2343-kserve-66497bf486-hkdbn"]
Sep 09 20:59:59.258297 ip-10-0-140-126 kubenswrapper[2550]: E0909 20:59:59.258278    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e\": container with ID starting with b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e not found: ID does not exist" containerID="b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e"
Sep 09 20:59:59.258350 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.258303    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e"} err="failed to get container status \"b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e\": rpc error: code = NotFound desc = could not find container \"b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e\": container with ID starting with b71cc09da9e761fe1e8c7f2d59f027c679881366501388f043876e640aa6c10e not found: ID does not exist"
Sep 09 20:59:59.268889 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.268871    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"]
Sep 09 20:59:59.275195 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.275159    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/embedding-w0-2b2343-kserve-67df8cdc5d-kdmjg"]
Sep 09 20:59:59.284980 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.284962    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"]
Sep 09 20:59:59.289393 ip-10-0-140-126 kubenswrapper[2550]: I0909 20:59:59.289373    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2b2343/simulator-w0-2b2343-kserve-66c5bc74f9-7nmm4"]
Sep 09 21:00:00.142421 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.142373    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j"]
Sep 09 21:00:00.143065 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143047    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="694b282b-0d69-4778-b2af-dd47c6b3c6b0" containerName="main"
Sep 09 21:00:00.143115 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143070    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="694b282b-0d69-4778-b2af-dd47c6b3c6b0" containerName="main"
Sep 09 21:00:00.143115 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143084    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy"
Sep 09 21:00:00.143115 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143093    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy"
Sep 09 21:00:00.143115 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143112    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6422b306-5913-4321-bdd5-e40ef4a961fa" containerName="payload-pre-processing"
Sep 09 21:00:00.143261 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143122    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="6422b306-5913-4321-bdd5-e40ef4a961fa" containerName="payload-pre-processing"
Sep 09 21:00:00.143261 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143142    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e45b1a2-815c-476d-980e-c505c810c987" containerName="main"
Sep 09 21:00:00.143261 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143151    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e45b1a2-815c-476d-980e-c505c810c987" containerName="main"
Sep 09 21:00:00.143261 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143196    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5e1b010-99bb-42de-94de-618e9cd83e91" containerName="main"
Sep 09 21:00:00.143261 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143202    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5e1b010-99bb-42de-94de-618e9cd83e91" containerName="main"
Sep 09 21:00:00.143404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143277    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5e1b010-99bb-42de-94de-618e9cd83e91" containerName="main"
Sep 09 21:00:00.143404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143288    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="6422b306-5913-4321-bdd5-e40ef4a961fa" containerName="payload-pre-processing"
Sep 09 21:00:00.143404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143296    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy"
Sep 09 21:00:00.143404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143302    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e45b1a2-815c-476d-980e-c505c810c987" containerName="main"
Sep 09 21:00:00.143404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143307    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="694b282b-0d69-4778-b2af-dd47c6b3c6b0" containerName="main"
Sep 09 21:00:00.143404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.143313    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy"
Sep 09 21:00:00.146291 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.146268    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j"
Sep 09 21:00:00.148915 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.148898    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tsks9\""
Sep 09 21:00:00.154996 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.154967    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j"]
Sep 09 21:00:00.159273 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.159252    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j"
Sep 09 21:00:00.166789 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.166713    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws"]
Sep 09 21:00:00.167560 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.167538    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy"
Sep 09 21:00:00.167560 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.167561    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" containerName="istio-proxy"
Sep 09 21:00:00.171416 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.171395    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws"
Sep 09 21:00:00.179469 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.179446    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws"]
Sep 09 21:00:00.201948 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.201804    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws"
Sep 09 21:00:00.282156 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.282123    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e45b1a2-815c-476d-980e-c505c810c987" path="/var/lib/kubelet/pods/5e45b1a2-815c-476d-980e-c505c810c987/volumes"
Sep 09 21:00:00.282801 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.282777    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="694b282b-0d69-4778-b2af-dd47c6b3c6b0" path="/var/lib/kubelet/pods/694b282b-0d69-4778-b2af-dd47c6b3c6b0/volumes"
Sep 09 21:00:00.283346 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.283326    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5e1b010-99bb-42de-94de-618e9cd83e91" path="/var/lib/kubelet/pods/a5e1b010-99bb-42de-94de-618e9cd83e91/volumes"
Sep 09 21:00:00.284364 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.284343    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae1828bc-806a-4e5b-8320-22fd7df34953" path="/var/lib/kubelet/pods/ae1828bc-806a-4e5b-8320-22fd7df34953/volumes"
Sep 09 21:00:00.448355 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.448326    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j"]
Sep 09 21:00:00.449464 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:00:00.449443    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6aabd45_004c_4f68_9991_b96e0a79c819.slice/crio-df926c38e1a10af59019e95000051ee6d7b5d531ace2e86dd1907e53e20c135b WatchSource:0}: Error finding container df926c38e1a10af59019e95000051ee6d7b5d531ace2e86dd1907e53e20c135b: Status 404 returned error can't find the container with id df926c38e1a10af59019e95000051ee6d7b5d531ace2e86dd1907e53e20c135b
Sep 09 21:00:00.485602 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:00.485576    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws"]
Sep 09 21:00:00.486220 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:00:00.486196    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7887e1f3_14d9_4a94_8531_bb59da1aae38.slice/crio-102de97135f7a000b2616ce86bf00e90b8762ff0c440a1e424282b3c52e3b3dd WatchSource:0}: Error finding container 102de97135f7a000b2616ce86bf00e90b8762ff0c440a1e424282b3c52e3b3dd: Status 404 returned error can't find the container with id 102de97135f7a000b2616ce86bf00e90b8762ff0c440a1e424282b3c52e3b3dd
Sep 09 21:00:01.240666 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:01.240626    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws" event={"ID":"7887e1f3-14d9-4a94-8531-bb59da1aae38","Type":"ContainerStarted","Data":"2a4a5f02a63696e724008e8616b3e2f401b07114c831842520543d002c0e1ac5"}
Sep 09 21:00:01.241122 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:01.240721    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws" event={"ID":"7887e1f3-14d9-4a94-8531-bb59da1aae38","Type":"ContainerStarted","Data":"102de97135f7a000b2616ce86bf00e90b8762ff0c440a1e424282b3c52e3b3dd"}
Sep 09 21:00:01.242667 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:01.242634    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j" event={"ID":"b6aabd45-004c-4f68-9991-b96e0a79c819","Type":"ContainerStarted","Data":"b24bca3d4d2947db25156203b0f049c1822465af31b82a56a46cd7c392a48791"}
Sep 09 21:00:01.242798 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:01.242690    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j" event={"ID":"b6aabd45-004c-4f68-9991-b96e0a79c819","Type":"ContainerStarted","Data":"df926c38e1a10af59019e95000051ee6d7b5d531ace2e86dd1907e53e20c135b"}
Sep 09 21:00:01.257040 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:01.256999    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws" podStartSLOduration=1.2569858680000001 podStartE2EDuration="1.256985868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:01.254263956 +0000 UTC m=+1661.404936299" watchObservedRunningTime="2026-09-09 21:00:01.256985868 +0000 UTC m=+1661.407658211"
Sep 09 21:00:01.271044 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:01.271009    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j" podStartSLOduration=1.270996802 podStartE2EDuration="1.270996802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:01.270428847 +0000 UTC m=+1661.421101190" watchObservedRunningTime="2026-09-09 21:00:01.270996802 +0000 UTC m=+1661.421669144"
Sep 09 21:00:02.065326 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:02.065293    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:02.207390 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:02.207355    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"]
Sep 09 21:00:02.207622 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:02.207575    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy" containerID="cri-o://73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba" gracePeriod=30
Sep 09 21:00:02.253722 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:02.253690    2550 generic.go:356] "Generic (PLEG): container finished" podID="7887e1f3-14d9-4a94-8531-bb59da1aae38" containerID="2a4a5f02a63696e724008e8616b3e2f401b07114c831842520543d002c0e1ac5" exitCode=0
Sep 09 21:00:02.254114 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:02.253780    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws" event={"ID":"7887e1f3-14d9-4a94-8531-bb59da1aae38","Type":"ContainerDied","Data":"2a4a5f02a63696e724008e8616b3e2f401b07114c831842520543d002c0e1ac5"}
Sep 09 21:00:02.255337 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:02.255317    2550 generic.go:356] "Generic (PLEG): container finished" podID="b6aabd45-004c-4f68-9991-b96e0a79c819" containerID="b24bca3d4d2947db25156203b0f049c1822465af31b82a56a46cd7c392a48791" exitCode=0
Sep 09 21:00:02.255418 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:02.255396    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j" event={"ID":"b6aabd45-004c-4f68-9991-b96e0a79c819","Type":"ContainerDied","Data":"b24bca3d4d2947db25156203b0f049c1822465af31b82a56a46cd7c392a48791"}
Sep 09 21:00:02.335213 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:02.335099    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:02.836971 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:02.836923    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:03.396706 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:03.396680    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws"
Sep 09 21:00:03.446228 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:03.446200    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j"
Sep 09 21:00:04.268790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:04.268765    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws"
Sep 09 21:00:04.268790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:04.268776    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws" event={"ID":"7887e1f3-14d9-4a94-8531-bb59da1aae38","Type":"ContainerDied","Data":"102de97135f7a000b2616ce86bf00e90b8762ff0c440a1e424282b3c52e3b3dd"}
Sep 09 21:00:04.268997 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:04.268831    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="102de97135f7a000b2616ce86bf00e90b8762ff0c440a1e424282b3c52e3b3dd"
Sep 09 21:00:04.270218 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:04.270193    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j"
Sep 09 21:00:04.270351 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:04.270217    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816460-sfl4j" event={"ID":"b6aabd45-004c-4f68-9991-b96e0a79c819","Type":"ContainerDied","Data":"df926c38e1a10af59019e95000051ee6d7b5d531ace2e86dd1907e53e20c135b"}
Sep 09 21:00:04.270351 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:04.270252    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df926c38e1a10af59019e95000051ee6d7b5d531ace2e86dd1907e53e20c135b"
Sep 09 21:00:04.706636 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:04.706549    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"]
Sep 09 21:00:04.707061 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:04.706769    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87" podUID="84634948-7d85-413e-bd88-238885c7cf16" containerName="payload-pre-processing" containerID="cri-o://3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9" gracePeriod=30
Sep 09 21:00:04.985856 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:04.985828    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 21:00:05.066021 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.065986    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:05.083507 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.083478    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84634948-7d85-413e-bd88-238885c7cf16-plugins-config-volume\") pod \"84634948-7d85-413e-bd88-238885c7cf16\" (UID: \"84634948-7d85-413e-bd88-238885c7cf16\") "
Sep 09 21:00:05.083671 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.083516    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85qqw\" (UniqueName: \"kubernetes.io/projected/84634948-7d85-413e-bd88-238885c7cf16-kube-api-access-85qqw\") pod \"84634948-7d85-413e-bd88-238885c7cf16\" (UID: \"84634948-7d85-413e-bd88-238885c7cf16\") "
Sep 09 21:00:05.083877 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.083850    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84634948-7d85-413e-bd88-238885c7cf16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84634948-7d85-413e-bd88-238885c7cf16" (UID: "84634948-7d85-413e-bd88-238885c7cf16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:00:05.085668 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.085643    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84634948-7d85-413e-bd88-238885c7cf16-kube-api-access-85qqw" (OuterVolumeSpecName: "kube-api-access-85qqw") pod "84634948-7d85-413e-bd88-238885c7cf16" (UID: "84634948-7d85-413e-bd88-238885c7cf16"). InnerVolumeSpecName "kube-api-access-85qqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:05.134604 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.134565    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:05.184933 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.184892    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84634948-7d85-413e-bd88-238885c7cf16-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:05.184933 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.184928    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85qqw\" (UniqueName: \"kubernetes.io/projected/84634948-7d85-413e-bd88-238885c7cf16-kube-api-access-85qqw\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:05.246327 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.246227    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:05.277596 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.277563    2550 generic.go:356] "Generic (PLEG): container finished" podID="84634948-7d85-413e-bd88-238885c7cf16" containerID="3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9" exitCode=0
Sep 09 21:00:05.277757 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.277598    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"
Sep 09 21:00:05.277757 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.277617    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87" event={"ID":"84634948-7d85-413e-bd88-238885c7cf16","Type":"ContainerDied","Data":"3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9"}
Sep 09 21:00:05.277757 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.277671    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87" event={"ID":"84634948-7d85-413e-bd88-238885c7cf16","Type":"ContainerDied","Data":"351c63388a9d2c68c68d43bebc62a1fc00f1f4240bcef9ae15316d7a6838a13a"}
Sep 09 21:00:05.277757 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.277703    2550 scope.go:117] "RemoveContainer" containerID="3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9"
Sep 09 21:00:05.288094 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.288076    2550 scope.go:117] "RemoveContainer" containerID="3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9"
Sep 09 21:00:05.288446 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:00:05.288425    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9\": container with ID starting with 3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9 not found: ID does not exist" containerID="3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9"
Sep 09 21:00:05.288508 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.288454    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9"} err="failed to get container status \"3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9\": rpc error: code = NotFound desc = could not find container \"3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9\": container with ID starting with 3248df98dc87bd4a3f10a661ec96e0c64dc0ed39b6fcdff1d6a461d00445b3a9 not found: ID does not exist"
Sep 09 21:00:05.301006 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.300984    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"]
Sep 09 21:00:05.304683 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:05.304664    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-78a7d5-5d65765d-rxc87"]
Sep 09 21:00:06.279547 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:06.279509    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84634948-7d85-413e-bd88-238885c7cf16" path="/var/lib/kubelet/pods/84634948-7d85-413e-bd88-238885c7cf16/volumes"
Sep 09 21:00:07.463928 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.463905    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 21:00:07.507554 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507487    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-certs\") pod \"ec1f47c1-105f-4110-8d27-9aee8344090a\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") "
Sep 09 21:00:07.507554 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507543    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-podinfo\") pod \"ec1f47c1-105f-4110-8d27-9aee8344090a\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") "
Sep 09 21:00:07.507738 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507577    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-envoy\") pod \"ec1f47c1-105f-4110-8d27-9aee8344090a\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") "
Sep 09 21:00:07.507738 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507619    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-socket\") pod \"ec1f47c1-105f-4110-8d27-9aee8344090a\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") "
Sep 09 21:00:07.507738 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507645    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-credential-socket\") pod \"ec1f47c1-105f-4110-8d27-9aee8344090a\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") "
Sep 09 21:00:07.507738 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507684    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-token\") pod \"ec1f47c1-105f-4110-8d27-9aee8344090a\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") "
Sep 09 21:00:07.507946 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507782    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec1f47c1-105f-4110-8d27-9aee8344090a" (UID: "ec1f47c1-105f-4110-8d27-9aee8344090a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:07.507946 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507869    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-data\") pod \"ec1f47c1-105f-4110-8d27-9aee8344090a\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") "
Sep 09 21:00:07.507946 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507911    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec1f47c1-105f-4110-8d27-9aee8344090a" (UID: "ec1f47c1-105f-4110-8d27-9aee8344090a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:07.507946 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507931    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbgl7\" (UniqueName: \"kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-kube-api-access-qbgl7\") pod \"ec1f47c1-105f-4110-8d27-9aee8344090a\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") "
Sep 09 21:00:07.508140 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507954    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec1f47c1-105f-4110-8d27-9aee8344090a" (UID: "ec1f47c1-105f-4110-8d27-9aee8344090a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:07.508140 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.507980    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1f47c1-105f-4110-8d27-9aee8344090a-istiod-ca-cert\") pod \"ec1f47c1-105f-4110-8d27-9aee8344090a\" (UID: \"ec1f47c1-105f-4110-8d27-9aee8344090a\") "
Sep 09 21:00:07.508140 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.508096    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec1f47c1-105f-4110-8d27-9aee8344090a" (UID: "ec1f47c1-105f-4110-8d27-9aee8344090a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:07.508717 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.508391    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:07.508717 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.508693    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:07.508717 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.508711    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:07.508946 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.508726    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:07.508946 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.508396    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec1f47c1-105f-4110-8d27-9aee8344090a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec1f47c1-105f-4110-8d27-9aee8344090a" (UID: "ec1f47c1-105f-4110-8d27-9aee8344090a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:00:07.510291 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.510259    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-kube-api-access-qbgl7" (OuterVolumeSpecName: "kube-api-access-qbgl7") pod "ec1f47c1-105f-4110-8d27-9aee8344090a" (UID: "ec1f47c1-105f-4110-8d27-9aee8344090a"). InnerVolumeSpecName "kube-api-access-qbgl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:07.510359 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.510300    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec1f47c1-105f-4110-8d27-9aee8344090a" (UID: "ec1f47c1-105f-4110-8d27-9aee8344090a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:07.510477 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.510452    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec1f47c1-105f-4110-8d27-9aee8344090a" (UID: "ec1f47c1-105f-4110-8d27-9aee8344090a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:07.510577 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.510493    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec1f47c1-105f-4110-8d27-9aee8344090a" (UID: "ec1f47c1-105f-4110-8d27-9aee8344090a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:00:07.609507 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.609478    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbgl7\" (UniqueName: \"kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-kube-api-access-qbgl7\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:07.609507 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.609504    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1f47c1-105f-4110-8d27-9aee8344090a-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:07.609682 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.609517    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:07.609682 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.609528    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:07.609682 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:07.609539    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1f47c1-105f-4110-8d27-9aee8344090a-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:08.297263 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.297232    2550 generic.go:356] "Generic (PLEG): container finished" podID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerID="73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba" exitCode=0
Sep 09 21:00:08.297394 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.297284    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"
Sep 09 21:00:08.297394 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.297320    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" event={"ID":"ec1f47c1-105f-4110-8d27-9aee8344090a","Type":"ContainerDied","Data":"73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba"}
Sep 09 21:00:08.297394 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.297379    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l" event={"ID":"ec1f47c1-105f-4110-8d27-9aee8344090a","Type":"ContainerDied","Data":"91c2d8d4f0281f4dc6d7be230791d108be6ebec2b5c2c0299a87e48fcf786749"}
Sep 09 21:00:08.297502 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.297402    2550 scope.go:117] "RemoveContainer" containerID="73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba"
Sep 09 21:00:08.306749 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.306731    2550 scope.go:117] "RemoveContainer" containerID="fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199"
Sep 09 21:00:08.315158 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.315123    2550 scope.go:117] "RemoveContainer" containerID="73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba"
Sep 09 21:00:08.315514 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:00:08.315482    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba\": container with ID starting with 73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba not found: ID does not exist" containerID="73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba"
Sep 09 21:00:08.315605 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.315535    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba"} err="failed to get container status \"73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba\": rpc error: code = NotFound desc = could not find container \"73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba\": container with ID starting with 73ebc07c9faebdb21ea134475a176e63ce80fb2e51586d4c2b857532c9d2a7ba not found: ID does not exist"
Sep 09 21:00:08.315605 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.315557    2550 scope.go:117] "RemoveContainer" containerID="fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199"
Sep 09 21:00:08.315845 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:00:08.315824    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199\": container with ID starting with fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199 not found: ID does not exist" containerID="fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199"
Sep 09 21:00:08.315911 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.315852    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199"} err="failed to get container status \"fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199\": rpc error: code = NotFound desc = could not find container \"fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199\": container with ID starting with fa09080c7d6671a282d5fc9dfa8a96f214541d68ffe6b257af7158066c4ba199 not found: ID does not exist"
Sep 09 21:00:08.316263 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.316248    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"]
Sep 09 21:00:08.318637 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:08.318618    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2b2343-openshift-default-6cbbc46f64-zbp6l"]
Sep 09 21:00:10.282368 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:10.282330    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" path="/var/lib/kubelet/pods/ec1f47c1-105f-4110-8d27-9aee8344090a/volumes"
Sep 09 21:00:11.441551 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:11.441510    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"]
Sep 09 21:00:11.442004 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:11.441755    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy" containerID="cri-o://e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc" gracePeriod=30
Sep 09 21:00:11.453082 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:11.453057    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:11.575577 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:11.575541    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:12.267476 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:12.267445    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:12.839692 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:12.839659    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:13.303391 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:13.303355    2550 patch_prober.go:28] interesting pod/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 21:00:13.303562 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:13.303410    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:00:13.669008 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:13.668929    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:16.695885 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.695863    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 21:00:16.800060 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800028    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-envoy\") pod \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") "
Sep 09 21:00:16.800219 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800073    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh8zx\" (UniqueName: \"kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-kube-api-access-kh8zx\") pod \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") "
Sep 09 21:00:16.800219 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800090    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-socket\") pod \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") "
Sep 09 21:00:16.800219 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800128    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istiod-ca-cert\") pod \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") "
Sep 09 21:00:16.800219 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800217    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-token\") pod \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") "
Sep 09 21:00:16.800408 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800246    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-data\") pod \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") "
Sep 09 21:00:16.800408 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800272    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-certs\") pod \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") "
Sep 09 21:00:16.800408 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800354    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a332682-7abd-4c7a-8fad-0e3af5df2b7b" (UID: "4a332682-7abd-4c7a-8fad-0e3af5df2b7b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:16.800570 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800475    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a332682-7abd-4c7a-8fad-0e3af5df2b7b" (UID: "4a332682-7abd-4c7a-8fad-0e3af5df2b7b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:16.800630 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800566    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-podinfo\") pod \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") "
Sep 09 21:00:16.800630 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800587    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a332682-7abd-4c7a-8fad-0e3af5df2b7b" (UID: "4a332682-7abd-4c7a-8fad-0e3af5df2b7b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:16.800630 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800624    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-credential-socket\") pod \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\" (UID: \"4a332682-7abd-4c7a-8fad-0e3af5df2b7b\") "
Sep 09 21:00:16.800849 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800622    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a332682-7abd-4c7a-8fad-0e3af5df2b7b" (UID: "4a332682-7abd-4c7a-8fad-0e3af5df2b7b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:00:16.800849 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800825    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a332682-7abd-4c7a-8fad-0e3af5df2b7b" (UID: "4a332682-7abd-4c7a-8fad-0e3af5df2b7b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:16.800996 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.800979    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:16.801043 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.801002    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:16.801043 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.801016    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:16.801043 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.801028    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:16.801043 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.801040    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:16.802315 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.802290    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-kube-api-access-kh8zx" (OuterVolumeSpecName: "kube-api-access-kh8zx") pod "4a332682-7abd-4c7a-8fad-0e3af5df2b7b" (UID: "4a332682-7abd-4c7a-8fad-0e3af5df2b7b"). InnerVolumeSpecName "kube-api-access-kh8zx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:16.802605 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.802589    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a332682-7abd-4c7a-8fad-0e3af5df2b7b" (UID: "4a332682-7abd-4c7a-8fad-0e3af5df2b7b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:16.802677 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.802651    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a332682-7abd-4c7a-8fad-0e3af5df2b7b" (UID: "4a332682-7abd-4c7a-8fad-0e3af5df2b7b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:16.802759 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.802738    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a332682-7abd-4c7a-8fad-0e3af5df2b7b" (UID: "4a332682-7abd-4c7a-8fad-0e3af5df2b7b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:00:16.902426 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.902398    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:16.902426 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.902423    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh8zx\" (UniqueName: \"kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-kube-api-access-kh8zx\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:16.902426 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.902433    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:16.902634 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:16.902443    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a332682-7abd-4c7a-8fad-0e3af5df2b7b-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:17.353376 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.353344    2550 generic.go:356] "Generic (PLEG): container finished" podID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerID="e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc" exitCode=0
Sep 09 21:00:17.353543 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.353392    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"
Sep 09 21:00:17.353543 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.353423    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" event={"ID":"4a332682-7abd-4c7a-8fad-0e3af5df2b7b","Type":"ContainerDied","Data":"e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc"}
Sep 09 21:00:17.353543 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.353474    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8" event={"ID":"4a332682-7abd-4c7a-8fad-0e3af5df2b7b","Type":"ContainerDied","Data":"0a183d152c9b8bbe564531c5d1dda207efa913296e5d5db01e2f3bac70a6ea80"}
Sep 09 21:00:17.353543 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.353486    2550 scope.go:117] "RemoveContainer" containerID="e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc"
Sep 09 21:00:17.363617 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.363600    2550 scope.go:117] "RemoveContainer" containerID="0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0"
Sep 09 21:00:17.372718 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.372697    2550 scope.go:117] "RemoveContainer" containerID="e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc"
Sep 09 21:00:17.373101 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:00:17.373068    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc\": container with ID starting with e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc not found: ID does not exist" containerID="e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc"
Sep 09 21:00:17.373193 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.373105    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc"} err="failed to get container status \"e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc\": rpc error: code = NotFound desc = could not find container \"e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc\": container with ID starting with e21a93a63697b56e72f08d39433fc46cb33d4ef29f8ddb1682b90c81a3cd34fc not found: ID does not exist"
Sep 09 21:00:17.373193 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.373129    2550 scope.go:117] "RemoveContainer" containerID="0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0"
Sep 09 21:00:17.373593 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:00:17.373571    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0\": container with ID starting with 0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0 not found: ID does not exist" containerID="0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0"
Sep 09 21:00:17.373654 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.373600    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0"} err="failed to get container status \"0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0\": rpc error: code = NotFound desc = could not find container \"0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0\": container with ID starting with 0574fa7160dc3d3d262f8fe0f880968828c7110e4b3b771c22c520f207aef5d0 not found: ID does not exist"
Sep 09 21:00:17.376149 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.376130    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"]
Sep 09 21:00:17.383285 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:17.383261    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-78a7d5-openshift-default-55c457ff6b-gzvs8"]
Sep 09 21:00:18.280471 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:18.280437    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" path="/var/lib/kubelet/pods/4a332682-7abd-4c7a-8fad-0e3af5df2b7b/volumes"
Sep 09 21:00:20.586528 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:20.586481    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:20.686894 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:20.686862    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:21.548583 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:21.548542    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:22.406237 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.406193    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:22.439364 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.439334    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"]
Sep 09 21:00:22.439581 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.439555    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq" podUID="3954a225-0a11-41a0-8ee6-321448e62e10" containerName="payload-processing" containerID="cri-o://6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c" gracePeriod=30
Sep 09 21:00:22.500831 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.500776    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:22.598193 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.598141    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:22.690807 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.690780    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 21:00:22.760686 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.760642    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:22.858356 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.858324    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrh9f\" (UniqueName: \"kubernetes.io/projected/3954a225-0a11-41a0-8ee6-321448e62e10-kube-api-access-zrh9f\") pod \"3954a225-0a11-41a0-8ee6-321448e62e10\" (UID: \"3954a225-0a11-41a0-8ee6-321448e62e10\") "
Sep 09 21:00:22.858512 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.858383    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3954a225-0a11-41a0-8ee6-321448e62e10-plugins-config-volume\") pod \"3954a225-0a11-41a0-8ee6-321448e62e10\" (UID: \"3954a225-0a11-41a0-8ee6-321448e62e10\") "
Sep 09 21:00:22.858738 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.858710    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3954a225-0a11-41a0-8ee6-321448e62e10-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3954a225-0a11-41a0-8ee6-321448e62e10" (UID: "3954a225-0a11-41a0-8ee6-321448e62e10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:00:22.860355 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.860327    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3954a225-0a11-41a0-8ee6-321448e62e10-kube-api-access-zrh9f" (OuterVolumeSpecName: "kube-api-access-zrh9f") pod "3954a225-0a11-41a0-8ee6-321448e62e10" (UID: "3954a225-0a11-41a0-8ee6-321448e62e10"). InnerVolumeSpecName "kube-api-access-zrh9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:22.959916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.959889    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrh9f\" (UniqueName: \"kubernetes.io/projected/3954a225-0a11-41a0-8ee6-321448e62e10-kube-api-access-zrh9f\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:22.959916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.959916    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3954a225-0a11-41a0-8ee6-321448e62e10-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:00:22.982508 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:22.982476    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:23.391634 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:23.391555    2550 generic.go:356] "Generic (PLEG): container finished" podID="3954a225-0a11-41a0-8ee6-321448e62e10" containerID="6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c" exitCode=0
Sep 09 21:00:23.391634 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:23.391599    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"
Sep 09 21:00:23.391634 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:23.391602    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq" event={"ID":"3954a225-0a11-41a0-8ee6-321448e62e10","Type":"ContainerDied","Data":"6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c"}
Sep 09 21:00:23.391861 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:23.391643    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq" event={"ID":"3954a225-0a11-41a0-8ee6-321448e62e10","Type":"ContainerDied","Data":"ae7f10693e1f5ffc3105958ef9471fdaad0946c45a7e5fcc2338ca98e5730acd"}
Sep 09 21:00:23.391861 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:23.391664    2550 scope.go:117] "RemoveContainer" containerID="6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c"
Sep 09 21:00:23.402803 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:23.402382    2550 scope.go:117] "RemoveContainer" containerID="6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c"
Sep 09 21:00:23.402803 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:00:23.402791    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c\": container with ID starting with 6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c not found: ID does not exist" containerID="6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c"
Sep 09 21:00:23.402925 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:23.402816    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c"} err="failed to get container status \"6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c\": rpc error: code = NotFound desc = could not find container \"6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c\": container with ID starting with 6412c1dbd3a65cb5b7e728075641ce0c26a779ea3d4ae05e9d8bc8dd25f1f26c not found: ID does not exist"
Sep 09 21:00:23.418431 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:23.418407    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"]
Sep 09 21:00:23.424146 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:23.424124    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3fa05f-dfffd6645-5lhzq"]
Sep 09 21:00:24.280753 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:24.280721    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3954a225-0a11-41a0-8ee6-321448e62e10" path="/var/lib/kubelet/pods/3954a225-0a11-41a0-8ee6-321448e62e10/volumes"
Sep 09 21:00:28.868724 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:28.868686    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:28.967221 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:28.967184    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:29.114771 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:29.114734    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:33.778400 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:33.778371    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:41.042021 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:00:41.041982    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:09.496425 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496388    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"]
Sep 09 21:01:09.496894 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496878    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy"
Sep 09 21:01:09.496945 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496897    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy"
Sep 09 21:01:09.496945 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496910    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3954a225-0a11-41a0-8ee6-321448e62e10" containerName="payload-processing"
Sep 09 21:01:09.496945 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496916    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3954a225-0a11-41a0-8ee6-321448e62e10" containerName="payload-processing"
Sep 09 21:01:09.496945 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496928    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy"
Sep 09 21:01:09.496945 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496934    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy"
Sep 09 21:01:09.496945 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496940    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy"
Sep 09 21:01:09.496945 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496945    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496953    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84634948-7d85-413e-bd88-238885c7cf16" containerName="payload-pre-processing"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496958    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="84634948-7d85-413e-bd88-238885c7cf16" containerName="payload-pre-processing"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496967    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6aabd45-004c-4f68-9991-b96e0a79c819" containerName="cleanup"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496972    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6aabd45-004c-4f68-9991-b96e0a79c819" containerName="cleanup"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496979    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7887e1f3-14d9-4a94-8531-bb59da1aae38" containerName="cleanup"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.496984    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="7887e1f3-14d9-4a94-8531-bb59da1aae38" containerName="cleanup"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.497047    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="84634948-7d85-413e-bd88-238885c7cf16" containerName="payload-pre-processing"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.497055    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.497064    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.497069    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a332682-7abd-4c7a-8fad-0e3af5df2b7b" containerName="istio-proxy"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.497075    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3954a225-0a11-41a0-8ee6-321448e62e10" containerName="payload-processing"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.497080    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="7887e1f3-14d9-4a94-8531-bb59da1aae38" containerName="cleanup"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.497085    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy"
Sep 09 21:01:09.497178 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.497090    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6aabd45-004c-4f68-9991-b96e0a79c819" containerName="cleanup"
Sep 09 21:01:09.500016 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.499993    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.502395 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.502369    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-933e9d\"/\"e2e-conflict-b-dff9f838-kserve-self-signed-certs\""
Sep 09 21:01:09.508680 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.508653    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"]
Sep 09 21:01:09.522304 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.522268    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:09.603290 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.603250    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72ndz\" (UniqueName: \"kubernetes.io/projected/a519e641-e288-4fb6-8cef-9970720283db-kube-api-access-72ndz\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.603462 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.603309    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-tmp-dir\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.603462 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.603374    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-dshm\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.603462 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.603420    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a519e641-e288-4fb6-8cef-9970720283db-tls-certs\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.603462 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.603454    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-home\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.603629 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.603482    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-model-cache\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.704656 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.704624    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72ndz\" (UniqueName: \"kubernetes.io/projected/a519e641-e288-4fb6-8cef-9970720283db-kube-api-access-72ndz\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.704827 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.704679    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-tmp-dir\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.704870 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.704854    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-dshm\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.704969 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.704952    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a519e641-e288-4fb6-8cef-9970720283db-tls-certs\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.705015 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.705001    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-home\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.705065 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.705050    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-model-cache\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.705111 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.705074    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-tmp-dir\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.705349 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.705330    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-home\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.705450 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.705398    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-model-cache\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.707003 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.706981    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-dshm\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.707269 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.707251    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a519e641-e288-4fb6-8cef-9970720283db-tls-certs\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.712507 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.712485    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72ndz\" (UniqueName: \"kubernetes.io/projected/a519e641-e288-4fb6-8cef-9970720283db-kube-api-access-72ndz\") pod \"e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") " pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:09.810416 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:09.810332    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:10.086760 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:10.086733    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"]
Sep 09 21:01:10.087720 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:01:10.087693    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda519e641_e288_4fb6_8cef_9970720283db.slice/crio-c2629667cd7df510509d7268c36ff73d7d62b448e1fde1631fbd4854581d920f WatchSource:0}: Error finding container c2629667cd7df510509d7268c36ff73d7d62b448e1fde1631fbd4854581d920f: Status 404 returned error can't find the container with id c2629667cd7df510509d7268c36ff73d7d62b448e1fde1631fbd4854581d920f
Sep 09 21:01:10.662425 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:10.662390    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l" event={"ID":"a519e641-e288-4fb6-8cef-9970720283db","Type":"ContainerStarted","Data":"e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4"}
Sep 09 21:01:10.662425 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:10.662431    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l" event={"ID":"a519e641-e288-4fb6-8cef-9970720283db","Type":"ContainerStarted","Data":"c2629667cd7df510509d7268c36ff73d7d62b448e1fde1631fbd4854581d920f"}
Sep 09 21:01:10.679654 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:10.679603    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l" podStartSLOduration=1.679586933 podStartE2EDuration="1.679586933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:10.678515891 +0000 UTC m=+1730.829188235" watchObservedRunningTime="2026-09-09 21:01:10.679586933 +0000 UTC m=+1730.830259276"
Sep 09 21:01:19.811119 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:19.811083    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:19.811636 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:19.811140    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:19.823841 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:19.823812    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:20.627214 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:20.627178    2550 scope.go:117] "RemoveContainer" containerID="8db306d87c170eb2da416209273c833c84187d4d4b736b82da22b147997ca952"
Sep 09 21:01:20.730064 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:20.730041    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:21.249137 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:21.249104    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:30.578916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:30.578881    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"]
Sep 09 21:01:30.591338 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:30.591293    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:32.920703 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:32.920668    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:45.599292 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.599231    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l" podUID="a519e641-e288-4fb6-8cef-9970720283db" containerName="main" containerID="cri-o://e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4" gracePeriod=45
Sep 09 21:01:45.850110 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.850053    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:45.865072 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.865041    2550 generic.go:356] "Generic (PLEG): container finished" podID="a519e641-e288-4fb6-8cef-9970720283db" containerID="e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4" exitCode=0
Sep 09 21:01:45.865202 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.865079    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"
Sep 09 21:01:45.865202 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.865120    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l" event={"ID":"a519e641-e288-4fb6-8cef-9970720283db","Type":"ContainerDied","Data":"e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4"}
Sep 09 21:01:45.865202 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.865176    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l" event={"ID":"a519e641-e288-4fb6-8cef-9970720283db","Type":"ContainerDied","Data":"c2629667cd7df510509d7268c36ff73d7d62b448e1fde1631fbd4854581d920f"}
Sep 09 21:01:45.865202 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.865200    2550 scope.go:117] "RemoveContainer" containerID="e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4"
Sep 09 21:01:45.878077 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.877593    2550 scope.go:117] "RemoveContainer" containerID="e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4"
Sep 09 21:01:45.878156 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:01:45.878062    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4\": container with ID starting with e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4 not found: ID does not exist" containerID="e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4"
Sep 09 21:01:45.878156 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.878099    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4"} err="failed to get container status \"e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4\": rpc error: code = NotFound desc = could not find container \"e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4\": container with ID starting with e7cf5e68e39d4c81863f98e5aec51412189997b17076d9ad519404ec582985f4 not found: ID does not exist"
Sep 09 21:01:45.960044 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960017    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-model-cache\") pod \"a519e641-e288-4fb6-8cef-9970720283db\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") "
Sep 09 21:01:45.960159 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960073    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72ndz\" (UniqueName: \"kubernetes.io/projected/a519e641-e288-4fb6-8cef-9970720283db-kube-api-access-72ndz\") pod \"a519e641-e288-4fb6-8cef-9970720283db\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") "
Sep 09 21:01:45.960159 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960128    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-home\") pod \"a519e641-e288-4fb6-8cef-9970720283db\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") "
Sep 09 21:01:45.960277 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960194    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-tmp-dir\") pod \"a519e641-e288-4fb6-8cef-9970720283db\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") "
Sep 09 21:01:45.960277 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960227    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-dshm\") pod \"a519e641-e288-4fb6-8cef-9970720283db\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") "
Sep 09 21:01:45.960277 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960250    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a519e641-e288-4fb6-8cef-9970720283db-tls-certs\") pod \"a519e641-e288-4fb6-8cef-9970720283db\" (UID: \"a519e641-e288-4fb6-8cef-9970720283db\") "
Sep 09 21:01:45.960433 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960314    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-model-cache" (OuterVolumeSpecName: "model-cache") pod "a519e641-e288-4fb6-8cef-9970720283db" (UID: "a519e641-e288-4fb6-8cef-9970720283db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:01:45.960492 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960458    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-home" (OuterVolumeSpecName: "home") pod "a519e641-e288-4fb6-8cef-9970720283db" (UID: "a519e641-e288-4fb6-8cef-9970720283db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:01:45.960492 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960467    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a519e641-e288-4fb6-8cef-9970720283db" (UID: "a519e641-e288-4fb6-8cef-9970720283db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:01:45.960593 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960564    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:01:45.960593 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960577    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:01:45.960593 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.960586    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:01:45.962394 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.962358    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-dshm" (OuterVolumeSpecName: "dshm") pod "a519e641-e288-4fb6-8cef-9970720283db" (UID: "a519e641-e288-4fb6-8cef-9970720283db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:01:45.962522 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.962491    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a519e641-e288-4fb6-8cef-9970720283db-kube-api-access-72ndz" (OuterVolumeSpecName: "kube-api-access-72ndz") pod "a519e641-e288-4fb6-8cef-9970720283db" (UID: "a519e641-e288-4fb6-8cef-9970720283db"). InnerVolumeSpecName "kube-api-access-72ndz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:01:45.962522 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:45.962506    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a519e641-e288-4fb6-8cef-9970720283db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a519e641-e288-4fb6-8cef-9970720283db" (UID: "a519e641-e288-4fb6-8cef-9970720283db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:01:46.060987 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:46.060964    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a519e641-e288-4fb6-8cef-9970720283db-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:01:46.060987 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:46.060988    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a519e641-e288-4fb6-8cef-9970720283db-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:01:46.061185 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:46.060997    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72ndz\" (UniqueName: \"kubernetes.io/projected/a519e641-e288-4fb6-8cef-9970720283db-kube-api-access-72ndz\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:01:46.188506 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:46.188475    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"]
Sep 09 21:01:46.190846 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:46.190827    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/e2e-conflict-b-dff9f838-kserve-59b989bdbf-m6f6l"]
Sep 09 21:01:46.280955 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:01:46.280929    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a519e641-e288-4fb6-8cef-9970720283db" path="/var/lib/kubelet/pods/a519e641-e288-4fb6-8cef-9970720283db/volumes"
Sep 09 21:02:10.300220 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:10.300157    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:20.429828 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:20.429792    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 21:02:20.436047 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:20.436022    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 21:02:38.280969 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:38.280932    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"]
Sep 09 21:02:38.302144 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:38.302089    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:39.795473 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:39.795439    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:43.326763 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:43.326721    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:53.301021 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.300935    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4" podUID="19ee405c-6630-4209-813f-7505eb1590a1" containerName="main" containerID="cri-o://72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886" gracePeriod=45
Sep 09 21:02:53.561403 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.561340    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 21:02:53.668286 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.668249    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19ee405c-6630-4209-813f-7505eb1590a1-tls-certs\") pod \"19ee405c-6630-4209-813f-7505eb1590a1\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") "
Sep 09 21:02:53.668445 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.668309    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvgjf\" (UniqueName: \"kubernetes.io/projected/19ee405c-6630-4209-813f-7505eb1590a1-kube-api-access-jvgjf\") pod \"19ee405c-6630-4209-813f-7505eb1590a1\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") "
Sep 09 21:02:53.668445 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.668348    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-home\") pod \"19ee405c-6630-4209-813f-7505eb1590a1\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") "
Sep 09 21:02:53.668445 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.668388    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-tmp-dir\") pod \"19ee405c-6630-4209-813f-7505eb1590a1\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") "
Sep 09 21:02:53.668445 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.668437    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-dshm\") pod \"19ee405c-6630-4209-813f-7505eb1590a1\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") "
Sep 09 21:02:53.668692 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.668463    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-model-cache\") pod \"19ee405c-6630-4209-813f-7505eb1590a1\" (UID: \"19ee405c-6630-4209-813f-7505eb1590a1\") "
Sep 09 21:02:53.668692 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.668642    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19ee405c-6630-4209-813f-7505eb1590a1" (UID: "19ee405c-6630-4209-813f-7505eb1590a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:02:53.668692 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.668679    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-home" (OuterVolumeSpecName: "home") pod "19ee405c-6630-4209-813f-7505eb1590a1" (UID: "19ee405c-6630-4209-813f-7505eb1590a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:02:53.668862 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.668742    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:02:53.668862 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.668781    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-model-cache" (OuterVolumeSpecName: "model-cache") pod "19ee405c-6630-4209-813f-7505eb1590a1" (UID: "19ee405c-6630-4209-813f-7505eb1590a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:02:53.670576 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.670552    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19ee405c-6630-4209-813f-7505eb1590a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19ee405c-6630-4209-813f-7505eb1590a1" (UID: "19ee405c-6630-4209-813f-7505eb1590a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:02:53.670695 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.670612    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-dshm" (OuterVolumeSpecName: "dshm") pod "19ee405c-6630-4209-813f-7505eb1590a1" (UID: "19ee405c-6630-4209-813f-7505eb1590a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:02:53.670695 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.670631    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19ee405c-6630-4209-813f-7505eb1590a1-kube-api-access-jvgjf" (OuterVolumeSpecName: "kube-api-access-jvgjf") pod "19ee405c-6630-4209-813f-7505eb1590a1" (UID: "19ee405c-6630-4209-813f-7505eb1590a1"). InnerVolumeSpecName "kube-api-access-jvgjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:02:53.769450 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.769417    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:02:53.769450 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.769443    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:02:53.769450 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.769454    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19ee405c-6630-4209-813f-7505eb1590a1-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:02:53.769688 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.769462    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvgjf\" (UniqueName: \"kubernetes.io/projected/19ee405c-6630-4209-813f-7505eb1590a1-kube-api-access-jvgjf\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:02:53.769688 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:53.769471    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19ee405c-6630-4209-813f-7505eb1590a1-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:02:54.260125 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:54.260087    2550 generic.go:356] "Generic (PLEG): container finished" podID="19ee405c-6630-4209-813f-7505eb1590a1" containerID="72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886" exitCode=0
Sep 09 21:02:54.260308 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:54.260128    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"
Sep 09 21:02:54.260308 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:54.260139    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4" event={"ID":"19ee405c-6630-4209-813f-7505eb1590a1","Type":"ContainerDied","Data":"72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886"}
Sep 09 21:02:54.260308 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:54.260204    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4" event={"ID":"19ee405c-6630-4209-813f-7505eb1590a1","Type":"ContainerDied","Data":"c23cb49e7764a3631b599b1f612ec16132976022276c63f44b980c2c94fe9871"}
Sep 09 21:02:54.260308 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:54.260224    2550 scope.go:117] "RemoveContainer" containerID="72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886"
Sep 09 21:02:54.270201 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:54.270182    2550 scope.go:117] "RemoveContainer" containerID="72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886"
Sep 09 21:02:54.270478 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:02:54.270459    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886\": container with ID starting with 72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886 not found: ID does not exist" containerID="72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886"
Sep 09 21:02:54.270522 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:54.270487    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886"} err="failed to get container status \"72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886\": rpc error: code = NotFound desc = could not find container \"72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886\": container with ID starting with 72e0568dafdf7386da837c43c18ebd7275a5a876d6eac733a2443bfd596dd886 not found: ID does not exist"
Sep 09 21:02:54.283902 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:54.283874    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"]
Sep 09 21:02:54.286655 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:54.286635    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/auth-test-model-cb82f2-kserve-785d78d47f-s4zq4"]
Sep 09 21:02:56.280620 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:02:56.280584    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19ee405c-6630-4209-813f-7505eb1590a1" path="/var/lib/kubelet/pods/19ee405c-6630-4209-813f-7505eb1590a1/volumes"
Sep 09 21:03:00.112002 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:00.111964    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:00.206936 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:00.206910    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:19.623894 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:19.623855    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:20.609581 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.609543    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"]
Sep 09 21:03:20.610120 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.610105    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy"
Sep 09 21:03:20.610184 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.610123    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec1f47c1-105f-4110-8d27-9aee8344090a" containerName="istio-proxy"
Sep 09 21:03:20.610184 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.610131    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19ee405c-6630-4209-813f-7505eb1590a1" containerName="main"
Sep 09 21:03:20.610184 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.610137    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="19ee405c-6630-4209-813f-7505eb1590a1" containerName="main"
Sep 09 21:03:20.610184 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.610144    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a519e641-e288-4fb6-8cef-9970720283db" containerName="main"
Sep 09 21:03:20.610184 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.610149    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a519e641-e288-4fb6-8cef-9970720283db" containerName="main"
Sep 09 21:03:20.610353 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.610239    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="19ee405c-6630-4209-813f-7505eb1590a1" containerName="main"
Sep 09 21:03:20.610353 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.610251    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a519e641-e288-4fb6-8cef-9970720283db" containerName="main"
Sep 09 21:03:20.614761 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.614745    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.617866 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.617842    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-a2fb5a\"/\"default-dockercfg-7v86l\""
Sep 09 21:03:20.618239 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.618214    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-a2fb5a\"/\"kube-root-ca.crt\""
Sep 09 21:03:20.618352 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.618337    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-a2fb5a\"/\"sub-test-model-a2fb5a-kserve-self-signed-certs\""
Sep 09 21:03:20.618475 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.618460    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-a2fb5a\"/\"openshift-service-ca.crt\""
Sep 09 21:03:20.625353 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.625332    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"]
Sep 09 21:03:20.652903 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.652866    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:20.711236 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.711203    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1529e5ef-f5e4-409d-b5cd-630e089bedca-tls-certs\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.711351 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.711253    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-tmp-dir\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.711351 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.711326    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-dshm\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.711446 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.711358    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgdqj\" (UniqueName: \"kubernetes.io/projected/1529e5ef-f5e4-409d-b5cd-630e089bedca-kube-api-access-pgdqj\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.711446 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.711411    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-home\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.711526 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.711457    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-model-cache\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.812542 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.812511    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-tmp-dir\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.812735 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.812556    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-dshm\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.812735 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.812588    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgdqj\" (UniqueName: \"kubernetes.io/projected/1529e5ef-f5e4-409d-b5cd-630e089bedca-kube-api-access-pgdqj\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.812735 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.812642    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-home\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.812735 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.812685    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-model-cache\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.812735 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.812713    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1529e5ef-f5e4-409d-b5cd-630e089bedca-tls-certs\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.813078 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.813055    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-tmp-dir\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.813219 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.813190    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-home\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.813294 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.813266    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-model-cache\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.814965 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.814938    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-dshm\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.815102 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.815084    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1529e5ef-f5e4-409d-b5cd-630e089bedca-tls-certs\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.820440 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.820420    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgdqj\" (UniqueName: \"kubernetes.io/projected/1529e5ef-f5e4-409d-b5cd-630e089bedca-kube-api-access-pgdqj\") pod \"sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:20.925860 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:20.925783    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:21.205142 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:21.205113    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"]
Sep 09 21:03:21.208118 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:03:21.208087    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1529e5ef_f5e4_409d_b5cd_630e089bedca.slice/crio-73c0bdc77fbca49d46d5bd2c3863095bf536353bd3f6ca6cbded8a89b9d395fa WatchSource:0}: Error finding container 73c0bdc77fbca49d46d5bd2c3863095bf536353bd3f6ca6cbded8a89b9d395fa: Status 404 returned error can't find the container with id 73c0bdc77fbca49d46d5bd2c3863095bf536353bd3f6ca6cbded8a89b9d395fa
Sep 09 21:03:21.415215 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:21.415173    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg" event={"ID":"1529e5ef-f5e4-409d-b5cd-630e089bedca","Type":"ContainerStarted","Data":"8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f"}
Sep 09 21:03:21.415384 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:21.415226    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg" event={"ID":"1529e5ef-f5e4-409d-b5cd-630e089bedca","Type":"ContainerStarted","Data":"73c0bdc77fbca49d46d5bd2c3863095bf536353bd3f6ca6cbded8a89b9d395fa"}
Sep 09 21:03:24.602943 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:24.602887    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg" podStartSLOduration=4.602870858 podStartE2EDuration="4.602870858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:21.435344212 +0000 UTC m=+1861.586016560" watchObservedRunningTime="2026-09-09 21:03:24.602870858 +0000 UTC m=+1864.753543201"
Sep 09 21:03:24.603494 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:24.603203    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"]
Sep 09 21:03:24.622056 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:24.621698    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"]
Sep 09 21:03:24.639850 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:24.639806    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"]
Sep 09 21:03:24.716587 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:24.716555    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:30.926682 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:30.926637    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:30.926682 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:30.926689    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:30.945045 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:30.945023    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:31.483272 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:31.483239    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:03:35.503859 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.503825    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"]
Sep 09 21:03:35.504363 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.504109    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw" podUID="4e608808-6c11-4bec-b102-0c8190042427" containerName="maas-api" containerID="cri-o://88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9" gracePeriod=30
Sep 09 21:03:35.554583 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.554544    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:35.574136 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.574112    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"]
Sep 09 21:03:35.574356 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.574331    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n" podUID="26957cab-ede5-4baf-ab09-1884410f4ed5" containerName="payload-processing" containerID="cri-o://2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68" gracePeriod=30
Sep 09 21:03:35.627639 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.627606    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:35.727990 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.727947    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:35.832594 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.832434    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:35.847639 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.847517    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 21:03:35.960446 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.960420    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwv6m\" (UniqueName: \"kubernetes.io/projected/26957cab-ede5-4baf-ab09-1884410f4ed5-kube-api-access-gwv6m\") pod \"26957cab-ede5-4baf-ab09-1884410f4ed5\" (UID: \"26957cab-ede5-4baf-ab09-1884410f4ed5\") "
Sep 09 21:03:35.960616 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.960521    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26957cab-ede5-4baf-ab09-1884410f4ed5-plugins-config-volume\") pod \"26957cab-ede5-4baf-ab09-1884410f4ed5\" (UID: \"26957cab-ede5-4baf-ab09-1884410f4ed5\") "
Sep 09 21:03:35.960886 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.960861    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26957cab-ede5-4baf-ab09-1884410f4ed5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26957cab-ede5-4baf-ab09-1884410f4ed5" (UID: "26957cab-ede5-4baf-ab09-1884410f4ed5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:03:35.962447 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.962417    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26957cab-ede5-4baf-ab09-1884410f4ed5-kube-api-access-gwv6m" (OuterVolumeSpecName: "kube-api-access-gwv6m") pod "26957cab-ede5-4baf-ab09-1884410f4ed5" (UID: "26957cab-ede5-4baf-ab09-1884410f4ed5"). InnerVolumeSpecName "kube-api-access-gwv6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:03:35.984285 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:35.984267    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 21:03:36.061573 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.061480    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e608808-6c11-4bec-b102-0c8190042427-openshift-service-ca\") pod \"4e608808-6c11-4bec-b102-0c8190042427\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") "
Sep 09 21:03:36.061573 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.061526    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtzwq\" (UniqueName: \"kubernetes.io/projected/4e608808-6c11-4bec-b102-0c8190042427-kube-api-access-vtzwq\") pod \"4e608808-6c11-4bec-b102-0c8190042427\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") "
Sep 09 21:03:36.061793 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.061582    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-metrics-certs\") pod \"4e608808-6c11-4bec-b102-0c8190042427\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") "
Sep 09 21:03:36.061793 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.061620    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-maas-api-tls\") pod \"4e608808-6c11-4bec-b102-0c8190042427\" (UID: \"4e608808-6c11-4bec-b102-0c8190042427\") "
Sep 09 21:03:36.061905 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.061869    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e608808-6c11-4bec-b102-0c8190042427-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4e608808-6c11-4bec-b102-0c8190042427" (UID: "4e608808-6c11-4bec-b102-0c8190042427"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:03:36.062045 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.062028    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e608808-6c11-4bec-b102-0c8190042427-openshift-service-ca\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:36.062109 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.062048    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26957cab-ede5-4baf-ab09-1884410f4ed5-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:36.062109 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.062057    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwv6m\" (UniqueName: \"kubernetes.io/projected/26957cab-ede5-4baf-ab09-1884410f4ed5-kube-api-access-gwv6m\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:36.064152 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.064122    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4e608808-6c11-4bec-b102-0c8190042427" (UID: "4e608808-6c11-4bec-b102-0c8190042427"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:03:36.064256 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.064221    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4e608808-6c11-4bec-b102-0c8190042427" (UID: "4e608808-6c11-4bec-b102-0c8190042427"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:03:36.064302 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.064266    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e608808-6c11-4bec-b102-0c8190042427-kube-api-access-vtzwq" (OuterVolumeSpecName: "kube-api-access-vtzwq") pod "4e608808-6c11-4bec-b102-0c8190042427" (UID: "4e608808-6c11-4bec-b102-0c8190042427"). InnerVolumeSpecName "kube-api-access-vtzwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:03:36.153572 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.153537    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:36.162723 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.162700    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtzwq\" (UniqueName: \"kubernetes.io/projected/4e608808-6c11-4bec-b102-0c8190042427-kube-api-access-vtzwq\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:36.162723 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.162722    2550 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-metrics-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:36.162872 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.162731    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e608808-6c11-4bec-b102-0c8190042427-maas-api-tls\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:36.501851 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.501817    2550 generic.go:356] "Generic (PLEG): container finished" podID="26957cab-ede5-4baf-ab09-1884410f4ed5" containerID="2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68" exitCode=0
Sep 09 21:03:36.502036 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.501866    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"
Sep 09 21:03:36.502036 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.501910    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n" event={"ID":"26957cab-ede5-4baf-ab09-1884410f4ed5","Type":"ContainerDied","Data":"2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68"}
Sep 09 21:03:36.502036 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.501969    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n" event={"ID":"26957cab-ede5-4baf-ab09-1884410f4ed5","Type":"ContainerDied","Data":"278062589ee623a36e9e9d21d0585ed71bed8b9be2514aa6d9def1324ea386de"}
Sep 09 21:03:36.502036 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.501997    2550 scope.go:117] "RemoveContainer" containerID="2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68"
Sep 09 21:03:36.503486 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.503463    2550 generic.go:356] "Generic (PLEG): container finished" podID="4e608808-6c11-4bec-b102-0c8190042427" containerID="88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9" exitCode=0
Sep 09 21:03:36.503608 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.503517    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"
Sep 09 21:03:36.503608 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.503556    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw" event={"ID":"4e608808-6c11-4bec-b102-0c8190042427","Type":"ContainerDied","Data":"88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9"}
Sep 09 21:03:36.503722 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.503607    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw" event={"ID":"4e608808-6c11-4bec-b102-0c8190042427","Type":"ContainerDied","Data":"776e39ecee10ddd5e8ef237aa8cd41b79ca9044a1e147964f54aa4431311d7ca"}
Sep 09 21:03:36.511345 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.511189    2550 scope.go:117] "RemoveContainer" containerID="2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68"
Sep 09 21:03:36.511537 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:03:36.511444    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68\": container with ID starting with 2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68 not found: ID does not exist" containerID="2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68"
Sep 09 21:03:36.511537 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.511474    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68"} err="failed to get container status \"2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68\": rpc error: code = NotFound desc = could not find container \"2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68\": container with ID starting with 2176a7b070b927c73a91d5163ab70707dcff4ef00eb4fce11e61547a1b195a68 not found: ID does not exist"
Sep 09 21:03:36.511537 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.511492    2550 scope.go:117] "RemoveContainer" containerID="88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9"
Sep 09 21:03:36.520019 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.519995    2550 scope.go:117] "RemoveContainer" containerID="88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9"
Sep 09 21:03:36.520138 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.520124    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"]
Sep 09 21:03:36.520340 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:03:36.520319    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9\": container with ID starting with 88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9 not found: ID does not exist" containerID="88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9"
Sep 09 21:03:36.520414 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.520348    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9"} err="failed to get container status \"88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9\": rpc error: code = NotFound desc = could not find container \"88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9\": container with ID starting with 88e0c0b916cd7c90c7cf03d7dca3b4a4b4c58aa4ba3c1a9a34164bc5652027e9 not found: ID does not exist"
Sep 09 21:03:36.523404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.523377    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-933e9d-765f6d6c57-8hz6n"]
Sep 09 21:03:36.534144 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.534121    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"]
Sep 09 21:03:36.539224 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:36.539204    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-933e9d-b85745f9c-2f5pw"]
Sep 09 21:03:38.279633 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:38.279598    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26957cab-ede5-4baf-ab09-1884410f4ed5" path="/var/lib/kubelet/pods/26957cab-ede5-4baf-ab09-1884410f4ed5/volumes"
Sep 09 21:03:38.279999 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:38.279987    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e608808-6c11-4bec-b102-0c8190042427" path="/var/lib/kubelet/pods/4e608808-6c11-4bec-b102-0c8190042427/volumes"
Sep 09 21:03:39.625469 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.625426    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm" podUID="fcaf2446-4001-4709-bf08-1c2a13170b77" containerName="main" containerID="cri-o://fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39" gracePeriod=45
Sep 09 21:03:39.643935 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.643877    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng" podUID="6e579cc1-f249-4168-8f97-4f1c5e3c241a" containerName="main" containerID="cri-o://d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8" gracePeriod=45
Sep 09 21:03:39.659373 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.659318    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j" podUID="2b733fc0-c554-4780-aa61-353b66ac36eb" containerName="main" containerID="cri-o://56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a" gracePeriod=45
Sep 09 21:03:39.944921 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.944895    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 21:03:39.956330 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.956306    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 21:03:39.970264 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.970234    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997230    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-home\") pod \"2b733fc0-c554-4780-aa61-353b66ac36eb\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997271    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-dshm\") pod \"fcaf2446-4001-4709-bf08-1c2a13170b77\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997299    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b733fc0-c554-4780-aa61-353b66ac36eb-tls-certs\") pod \"2b733fc0-c554-4780-aa61-353b66ac36eb\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997334    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-model-cache\") pod \"fcaf2446-4001-4709-bf08-1c2a13170b77\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997386    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-tmp-dir\") pod \"2b733fc0-c554-4780-aa61-353b66ac36eb\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997437    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-model-cache\") pod \"2b733fc0-c554-4780-aa61-353b66ac36eb\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997471    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-tmp-dir\") pod \"fcaf2446-4001-4709-bf08-1c2a13170b77\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997507    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcaf2446-4001-4709-bf08-1c2a13170b77-tls-certs\") pod \"fcaf2446-4001-4709-bf08-1c2a13170b77\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997523    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-home" (OuterVolumeSpecName: "home") pod "2b733fc0-c554-4780-aa61-353b66ac36eb" (UID: "2b733fc0-c554-4780-aa61-353b66ac36eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997538    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7b7gd\" (UniqueName: \"kubernetes.io/projected/fcaf2446-4001-4709-bf08-1c2a13170b77-kube-api-access-7b7gd\") pod \"fcaf2446-4001-4709-bf08-1c2a13170b77\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997605    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwzs5\" (UniqueName: \"kubernetes.io/projected/2b733fc0-c554-4780-aa61-353b66ac36eb-kube-api-access-kwzs5\") pod \"2b733fc0-c554-4780-aa61-353b66ac36eb\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997666    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-home\") pod \"fcaf2446-4001-4709-bf08-1c2a13170b77\" (UID: \"fcaf2446-4001-4709-bf08-1c2a13170b77\") "
Sep 09 21:03:39.997916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.997698    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-dshm\") pod \"2b733fc0-c554-4780-aa61-353b66ac36eb\" (UID: \"2b733fc0-c554-4780-aa61-353b66ac36eb\") "
Sep 09 21:03:39.998728 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.998056    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:39.998728 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.998574    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b733fc0-c554-4780-aa61-353b66ac36eb" (UID: "2b733fc0-c554-4780-aa61-353b66ac36eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:39.998847 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.998757    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b733fc0-c554-4780-aa61-353b66ac36eb" (UID: "2b733fc0-c554-4780-aa61-353b66ac36eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:39.999288 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.998930    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fcaf2446-4001-4709-bf08-1c2a13170b77" (UID: "fcaf2446-4001-4709-bf08-1c2a13170b77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:39.999288 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.999258    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-home" (OuterVolumeSpecName: "home") pod "fcaf2446-4001-4709-bf08-1c2a13170b77" (UID: "fcaf2446-4001-4709-bf08-1c2a13170b77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:39.999542 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:39.999513    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-model-cache" (OuterVolumeSpecName: "model-cache") pod "fcaf2446-4001-4709-bf08-1c2a13170b77" (UID: "fcaf2446-4001-4709-bf08-1c2a13170b77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:40.001567 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.001532    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-dshm" (OuterVolumeSpecName: "dshm") pod "2b733fc0-c554-4780-aa61-353b66ac36eb" (UID: "2b733fc0-c554-4780-aa61-353b66ac36eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:40.002644 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.002608    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b733fc0-c554-4780-aa61-353b66ac36eb-kube-api-access-kwzs5" (OuterVolumeSpecName: "kube-api-access-kwzs5") pod "2b733fc0-c554-4780-aa61-353b66ac36eb" (UID: "2b733fc0-c554-4780-aa61-353b66ac36eb"). InnerVolumeSpecName "kube-api-access-kwzs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:03:40.003188 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.003133    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b733fc0-c554-4780-aa61-353b66ac36eb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b733fc0-c554-4780-aa61-353b66ac36eb" (UID: "2b733fc0-c554-4780-aa61-353b66ac36eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:03:40.003716 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.003659    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-dshm" (OuterVolumeSpecName: "dshm") pod "fcaf2446-4001-4709-bf08-1c2a13170b77" (UID: "fcaf2446-4001-4709-bf08-1c2a13170b77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:40.003810 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.003752    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcaf2446-4001-4709-bf08-1c2a13170b77-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fcaf2446-4001-4709-bf08-1c2a13170b77" (UID: "fcaf2446-4001-4709-bf08-1c2a13170b77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:03:40.004194 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.004143    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcaf2446-4001-4709-bf08-1c2a13170b77-kube-api-access-7b7gd" (OuterVolumeSpecName: "kube-api-access-7b7gd") pod "fcaf2446-4001-4709-bf08-1c2a13170b77" (UID: "fcaf2446-4001-4709-bf08-1c2a13170b77"). InnerVolumeSpecName "kube-api-access-7b7gd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:03:40.098653 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.098625    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tmp-dir\") pod \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") "
Sep 09 21:03:40.098843 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.098669    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-model-cache\") pod \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") "
Sep 09 21:03:40.098843 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.098688    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-home\") pod \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") "
Sep 09 21:03:40.098843 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.098731    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzxp6\" (UniqueName: \"kubernetes.io/projected/6e579cc1-f249-4168-8f97-4f1c5e3c241a-kube-api-access-bzxp6\") pod \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") "
Sep 09 21:03:40.098843 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.098759    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-dshm\") pod \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") "
Sep 09 21:03:40.099072 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.098872    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6e579cc1-f249-4168-8f97-4f1c5e3c241a" (UID: "6e579cc1-f249-4168-8f97-4f1c5e3c241a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:40.099072 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.098922    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-model-cache" (OuterVolumeSpecName: "model-cache") pod "6e579cc1-f249-4168-8f97-4f1c5e3c241a" (UID: "6e579cc1-f249-4168-8f97-4f1c5e3c241a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:40.099072 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.098930    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tls-certs\") pod \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\" (UID: \"6e579cc1-f249-4168-8f97-4f1c5e3c241a\") "
Sep 09 21:03:40.099072 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099003    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-home" (OuterVolumeSpecName: "home") pod "6e579cc1-f249-4168-8f97-4f1c5e3c241a" (UID: "6e579cc1-f249-4168-8f97-4f1c5e3c241a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:40.099330 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099313    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099396 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099338    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099396 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099354    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099396 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099367    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099396 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099379    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099396 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099391    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099598 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099403    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b733fc0-c554-4780-aa61-353b66ac36eb-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099598 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099416    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099598 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099429    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099598 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099441    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b733fc0-c554-4780-aa61-353b66ac36eb-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099598 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099454    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcaf2446-4001-4709-bf08-1c2a13170b77-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099598 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099466    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcaf2446-4001-4709-bf08-1c2a13170b77-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099598 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099480    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b7gd\" (UniqueName: \"kubernetes.io/projected/fcaf2446-4001-4709-bf08-1c2a13170b77-kube-api-access-7b7gd\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.099598 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.099492    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwzs5\" (UniqueName: \"kubernetes.io/projected/2b733fc0-c554-4780-aa61-353b66ac36eb-kube-api-access-kwzs5\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.100936 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.100910    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-dshm" (OuterVolumeSpecName: "dshm") pod "6e579cc1-f249-4168-8f97-4f1c5e3c241a" (UID: "6e579cc1-f249-4168-8f97-4f1c5e3c241a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:40.101020 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.100953    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e579cc1-f249-4168-8f97-4f1c5e3c241a-kube-api-access-bzxp6" (OuterVolumeSpecName: "kube-api-access-bzxp6") pod "6e579cc1-f249-4168-8f97-4f1c5e3c241a" (UID: "6e579cc1-f249-4168-8f97-4f1c5e3c241a"). InnerVolumeSpecName "kube-api-access-bzxp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:03:40.101020 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.100967    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6e579cc1-f249-4168-8f97-4f1c5e3c241a" (UID: "6e579cc1-f249-4168-8f97-4f1c5e3c241a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:03:40.200285 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.200210    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e579cc1-f249-4168-8f97-4f1c5e3c241a-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.200285 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.200236    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzxp6\" (UniqueName: \"kubernetes.io/projected/6e579cc1-f249-4168-8f97-4f1c5e3c241a-kube-api-access-bzxp6\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.200285 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.200248    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e579cc1-f249-4168-8f97-4f1c5e3c241a-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:40.531618 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.531581    2550 generic.go:356] "Generic (PLEG): container finished" podID="2b733fc0-c554-4780-aa61-353b66ac36eb" containerID="56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a" exitCode=0
Sep 09 21:03:40.531803 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.531621    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"
Sep 09 21:03:40.531803 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.531657    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j" event={"ID":"2b733fc0-c554-4780-aa61-353b66ac36eb","Type":"ContainerDied","Data":"56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a"}
Sep 09 21:03:40.531803 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.531699    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j" event={"ID":"2b733fc0-c554-4780-aa61-353b66ac36eb","Type":"ContainerDied","Data":"0d0b906a6653156665466241486e22476e331dab2f1fb4333d23ea556462f2ee"}
Sep 09 21:03:40.531803 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.531721    2550 scope.go:117] "RemoveContainer" containerID="56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a"
Sep 09 21:03:40.533299 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.533273    2550 generic.go:356] "Generic (PLEG): container finished" podID="6e579cc1-f249-4168-8f97-4f1c5e3c241a" containerID="d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8" exitCode=0
Sep 09 21:03:40.533418 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.533321    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"
Sep 09 21:03:40.533418 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.533357    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng" event={"ID":"6e579cc1-f249-4168-8f97-4f1c5e3c241a","Type":"ContainerDied","Data":"d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8"}
Sep 09 21:03:40.533418 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.533389    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng" event={"ID":"6e579cc1-f249-4168-8f97-4f1c5e3c241a","Type":"ContainerDied","Data":"a4e462ca4221333f277baab11b4fa400c29e81bdb6c5a5888622c540c794a9b0"}
Sep 09 21:03:40.535248 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.535225    2550 generic.go:356] "Generic (PLEG): container finished" podID="fcaf2446-4001-4709-bf08-1c2a13170b77" containerID="fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39" exitCode=0
Sep 09 21:03:40.535372 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.535258    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"
Sep 09 21:03:40.535372 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.535300    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm" event={"ID":"fcaf2446-4001-4709-bf08-1c2a13170b77","Type":"ContainerDied","Data":"fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39"}
Sep 09 21:03:40.535372 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.535331    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm" event={"ID":"fcaf2446-4001-4709-bf08-1c2a13170b77","Type":"ContainerDied","Data":"85df44c238607d782f9ef8b51663f517ebcc60ca770f5f4ad6e6d8406473f9e4"}
Sep 09 21:03:40.542368 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.542351    2550 scope.go:117] "RemoveContainer" containerID="56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a"
Sep 09 21:03:40.542669 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:03:40.542652    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a\": container with ID starting with 56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a not found: ID does not exist" containerID="56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a"
Sep 09 21:03:40.542752 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.542682    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a"} err="failed to get container status \"56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a\": rpc error: code = NotFound desc = could not find container \"56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a\": container with ID starting with 56b689bbdeb7e6fdf8ba71b1e60c3898f9cc37cf437123f362f26f4c83bfc38a not found: ID does not exist"
Sep 09 21:03:40.542752 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.542705    2550 scope.go:117] "RemoveContainer" containerID="d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8"
Sep 09 21:03:40.551540 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.551520    2550 scope.go:117] "RemoveContainer" containerID="d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8"
Sep 09 21:03:40.551803 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:03:40.551786    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8\": container with ID starting with d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8 not found: ID does not exist" containerID="d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8"
Sep 09 21:03:40.551884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.551806    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8"} err="failed to get container status \"d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8\": rpc error: code = NotFound desc = could not find container \"d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8\": container with ID starting with d70a343e6e08636e503f7d6ac860da405fddc9618a525d3b8ca594f5ca3088d8 not found: ID does not exist"
Sep 09 21:03:40.551884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.551822    2550 scope.go:117] "RemoveContainer" containerID="fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39"
Sep 09 21:03:40.553078 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.553061    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"]
Sep 09 21:03:40.558584 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.558547    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/simulator-w4-933e9d-kserve-67f88dd894-lvd4j"]
Sep 09 21:03:40.559952 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.559931    2550 scope.go:117] "RemoveContainer" containerID="fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39"
Sep 09 21:03:40.560206 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:03:40.560184    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39\": container with ID starting with fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39 not found: ID does not exist" containerID="fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39"
Sep 09 21:03:40.560294 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.560209    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39"} err="failed to get container status \"fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39\": rpc error: code = NotFound desc = could not find container \"fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39\": container with ID starting with fe6c84fff63325c432cc75198103874b6d04fba09e2c2272df2acee3b8edab39 not found: ID does not exist"
Sep 09 21:03:40.567706 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.567687    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"]
Sep 09 21:03:40.571589 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.571571    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/embedding-w4-933e9d-kserve-6cdcd7ddbc-w6jng"]
Sep 09 21:03:40.580064 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.580045    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"]
Sep 09 21:03:40.584232 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:40.584215    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-933e9d/distinct-2-w4-933e9d-kserve-5849f96745-6bqcm"]
Sep 09 21:03:42.279332 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:42.279301    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b733fc0-c554-4780-aa61-353b66ac36eb" path="/var/lib/kubelet/pods/2b733fc0-c554-4780-aa61-353b66ac36eb/volumes"
Sep 09 21:03:42.279816 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:42.279686    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e579cc1-f249-4168-8f97-4f1c5e3c241a" path="/var/lib/kubelet/pods/6e579cc1-f249-4168-8f97-4f1c5e3c241a/volumes"
Sep 09 21:03:42.280020 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:42.280007    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcaf2446-4001-4709-bf08-1c2a13170b77" path="/var/lib/kubelet/pods/fcaf2446-4001-4709-bf08-1c2a13170b77/volumes"
Sep 09 21:03:42.387881 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:42.387845    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:42.490793 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:42.490762    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:42.630911 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:42.630818    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"]
Sep 09 21:03:42.631355 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:42.631090    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy" containerID="cri-o://199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d" gracePeriod=30
Sep 09 21:03:42.640380 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:42.640354    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:47.883795 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.883772    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 21:03:47.976713 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.976688    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-token\") pod \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") "
Sep 09 21:03:47.976862 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.976730    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-socket\") pod \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") "
Sep 09 21:03:47.976862 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.976772    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qdsq\" (UniqueName: \"kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-kube-api-access-5qdsq\") pod \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") "
Sep 09 21:03:47.976862 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.976796    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-podinfo\") pod \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") "
Sep 09 21:03:47.976862 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.976851    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-envoy\") pod \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") "
Sep 09 21:03:47.977080 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.976877    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-certs\") pod \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") "
Sep 09 21:03:47.977080 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.976929    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-credential-socket\") pod \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") "
Sep 09 21:03:47.977080 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.977013    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-data\") pod \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") "
Sep 09 21:03:47.977080 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.977039    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istiod-ca-cert\") pod \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\" (UID: \"e7f3bc29-b848-4d1b-8fe5-f06925a830de\") "
Sep 09 21:03:47.977337 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.977077    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e7f3bc29-b848-4d1b-8fe5-f06925a830de" (UID: "e7f3bc29-b848-4d1b-8fe5-f06925a830de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:47.977337 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.977295    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-data" (OuterVolumeSpecName: "istio-data") pod "e7f3bc29-b848-4d1b-8fe5-f06925a830de" (UID: "e7f3bc29-b848-4d1b-8fe5-f06925a830de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:47.977452 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.977386    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:47.977452 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.977403    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:47.977452 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.977412    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e7f3bc29-b848-4d1b-8fe5-f06925a830de" (UID: "e7f3bc29-b848-4d1b-8fe5-f06925a830de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:47.977611 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.977486    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e7f3bc29-b848-4d1b-8fe5-f06925a830de" (UID: "e7f3bc29-b848-4d1b-8fe5-f06925a830de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:47.977611 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.977565    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e7f3bc29-b848-4d1b-8fe5-f06925a830de" (UID: "e7f3bc29-b848-4d1b-8fe5-f06925a830de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:03:47.978999 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.978979    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e7f3bc29-b848-4d1b-8fe5-f06925a830de" (UID: "e7f3bc29-b848-4d1b-8fe5-f06925a830de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:03:47.979089 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.979009    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-kube-api-access-5qdsq" (OuterVolumeSpecName: "kube-api-access-5qdsq") pod "e7f3bc29-b848-4d1b-8fe5-f06925a830de" (UID: "e7f3bc29-b848-4d1b-8fe5-f06925a830de"). InnerVolumeSpecName "kube-api-access-5qdsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:03:47.979296 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.979273    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7f3bc29-b848-4d1b-8fe5-f06925a830de" (UID: "e7f3bc29-b848-4d1b-8fe5-f06925a830de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:03:47.979397 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:47.979269    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e7f3bc29-b848-4d1b-8fe5-f06925a830de" (UID: "e7f3bc29-b848-4d1b-8fe5-f06925a830de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:48.078104 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.078073    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:48.078104 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.078100    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:48.078104 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.078109    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7f3bc29-b848-4d1b-8fe5-f06925a830de-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:48.078347 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.078119    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:48.078347 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.078127    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:48.078347 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.078134    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qdsq\" (UniqueName: \"kubernetes.io/projected/e7f3bc29-b848-4d1b-8fe5-f06925a830de-kube-api-access-5qdsq\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:48.078347 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.078142    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7f3bc29-b848-4d1b-8fe5-f06925a830de-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:03:48.586232 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.586140    2550 generic.go:356] "Generic (PLEG): container finished" podID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerID="199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d" exitCode=0
Sep 09 21:03:48.586232 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.586197    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"
Sep 09 21:03:48.586414 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.586232    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" event={"ID":"e7f3bc29-b848-4d1b-8fe5-f06925a830de","Type":"ContainerDied","Data":"199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d"}
Sep 09 21:03:48.586414 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.586287    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms" event={"ID":"e7f3bc29-b848-4d1b-8fe5-f06925a830de","Type":"ContainerDied","Data":"aafe9d9f13cb180275625ac682e35b05ae359cbc68bc1fd96b52b88b6a90a7e0"}
Sep 09 21:03:48.586414 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.586309    2550 scope.go:117] "RemoveContainer" containerID="199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d"
Sep 09 21:03:48.595346 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.595331    2550 scope.go:117] "RemoveContainer" containerID="0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a"
Sep 09 21:03:48.603418 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.603399    2550 scope.go:117] "RemoveContainer" containerID="199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d"
Sep 09 21:03:48.603676 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:03:48.603655    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d\": container with ID starting with 199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d not found: ID does not exist" containerID="199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d"
Sep 09 21:03:48.603734 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.603687    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d"} err="failed to get container status \"199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d\": rpc error: code = NotFound desc = could not find container \"199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d\": container with ID starting with 199e537ec8f38126a44a93ccfd95955283a33410f16a065b27f2d5d71548138d not found: ID does not exist"
Sep 09 21:03:48.603734 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.603705    2550 scope.go:117] "RemoveContainer" containerID="0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a"
Sep 09 21:03:48.603967 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:03:48.603949    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a\": container with ID starting with 0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a not found: ID does not exist" containerID="0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a"
Sep 09 21:03:48.604017 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.603970    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a"} err="failed to get container status \"0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a\": rpc error: code = NotFound desc = could not find container \"0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a\": container with ID starting with 0626592e85911e44b70fbf1daf436bbb0a602e54290c3f939586fedc6bd9ac3a not found: ID does not exist"
Sep 09 21:03:48.604459 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.604438    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"]
Sep 09 21:03:48.608288 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:48.608270    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-933e9d-openshift-default-67976b7cc8-pknms"]
Sep 09 21:03:50.286308 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:03:50.286267    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" path="/var/lib/kubelet/pods/e7f3bc29-b848-4d1b-8fe5-f06925a830de/volumes"
Sep 09 21:04:06.502503 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:06.502464    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:06.603476 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:06.603444    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:07.484517 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484483    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7675-a84178"]
Sep 09 21:04:07.484839 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484827    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy"
Sep 09 21:04:07.484884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484841    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy"
Sep 09 21:04:07.484884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484850    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26957cab-ede5-4baf-ab09-1884410f4ed5" containerName="payload-processing"
Sep 09 21:04:07.484884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484855    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="26957cab-ede5-4baf-ab09-1884410f4ed5" containerName="payload-processing"
Sep 09 21:04:07.484884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484862    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b733fc0-c554-4780-aa61-353b66ac36eb" containerName="main"
Sep 09 21:04:07.484884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484868    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b733fc0-c554-4780-aa61-353b66ac36eb" containerName="main"
Sep 09 21:04:07.484884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484873    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcaf2446-4001-4709-bf08-1c2a13170b77" containerName="main"
Sep 09 21:04:07.484884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484878    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcaf2446-4001-4709-bf08-1c2a13170b77" containerName="main"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484889    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e608808-6c11-4bec-b102-0c8190042427" containerName="maas-api"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484894    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e608808-6c11-4bec-b102-0c8190042427" containerName="maas-api"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484913    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e579cc1-f249-4168-8f97-4f1c5e3c241a" containerName="main"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484917    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e579cc1-f249-4168-8f97-4f1c5e3c241a" containerName="main"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484968    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e608808-6c11-4bec-b102-0c8190042427" containerName="maas-api"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484975    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="26957cab-ede5-4baf-ab09-1884410f4ed5" containerName="payload-processing"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484981    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b733fc0-c554-4780-aa61-353b66ac36eb" containerName="main"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484986    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484993    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.484999    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e579cc1-f249-4168-8f97-4f1c5e3c241a" containerName="main"
Sep 09 21:04:07.485103 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.485005    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcaf2446-4001-4709-bf08-1c2a13170b77" containerName="main"
Sep 09 21:04:07.488036 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.488021    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7675-a84178"
Sep 09 21:04:07.490408 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.490386    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-744w4\""
Sep 09 21:04:07.496751 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.496726    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7675-a84178"]
Sep 09 21:04:07.559421 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.559392    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfkrl\" (UniqueName: \"kubernetes.io/projected/40888f18-1d7d-4d38-983c-ec6e50e20c80-kube-api-access-wfkrl\") pod \"mt-curl-7675-a84178\" (UID: \"40888f18-1d7d-4d38-983c-ec6e50e20c80\") " pod="opendatahub/mt-curl-7675-a84178"
Sep 09 21:04:07.660523 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.660489    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfkrl\" (UniqueName: \"kubernetes.io/projected/40888f18-1d7d-4d38-983c-ec6e50e20c80-kube-api-access-wfkrl\") pod \"mt-curl-7675-a84178\" (UID: \"40888f18-1d7d-4d38-983c-ec6e50e20c80\") " pod="opendatahub/mt-curl-7675-a84178"
Sep 09 21:04:07.672445 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.672416    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfkrl\" (UniqueName: \"kubernetes.io/projected/40888f18-1d7d-4d38-983c-ec6e50e20c80-kube-api-access-wfkrl\") pod \"mt-curl-7675-a84178\" (UID: \"40888f18-1d7d-4d38-983c-ec6e50e20c80\") " pod="opendatahub/mt-curl-7675-a84178"
Sep 09 21:04:07.798729 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:07.798611    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7675-a84178"
Sep 09 21:04:08.072879 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:08.072849    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7675-a84178"]
Sep 09 21:04:08.074486 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:04:08.074459    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40888f18_1d7d_4d38_983c_ec6e50e20c80.slice/crio-13c75620c4d3c9adc6414ea895cfd32c856401f674994736c5d9a2e4933ab1b0 WatchSource:0}: Error finding container 13c75620c4d3c9adc6414ea895cfd32c856401f674994736c5d9a2e4933ab1b0: Status 404 returned error can't find the container with id 13c75620c4d3c9adc6414ea895cfd32c856401f674994736c5d9a2e4933ab1b0
Sep 09 21:04:08.075774 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:08.075755    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 21:04:08.359727 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:08.359645    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:08.705369 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:08.705295    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7675-a84178" event={"ID":"40888f18-1d7d-4d38-983c-ec6e50e20c80","Type":"ContainerStarted","Data":"13c75620c4d3c9adc6414ea895cfd32c856401f674994736c5d9a2e4933ab1b0"}
Sep 09 21:04:09.712327 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:09.712293    2550 generic.go:356] "Generic (PLEG): container finished" podID="40888f18-1d7d-4d38-983c-ec6e50e20c80" containerID="494c9eb865ce1d19c5502489d288d862cb5f1fe35076a6d3367d9e67ddbe4718" exitCode=0
Sep 09 21:04:09.712736 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:09.712383    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7675-a84178" event={"ID":"40888f18-1d7d-4d38-983c-ec6e50e20c80","Type":"ContainerDied","Data":"494c9eb865ce1d19c5502489d288d862cb5f1fe35076a6d3367d9e67ddbe4718"}
Sep 09 21:04:10.852568 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:10.852543    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7675-a84178"
Sep 09 21:04:10.874878 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:10.874848    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7675-a84178_40888f18-1d7d-4d38-983c-ec6e50e20c80/mt-curl-7675-a84178/0.log"
Sep 09 21:04:10.891212 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:10.891182    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfkrl\" (UniqueName: \"kubernetes.io/projected/40888f18-1d7d-4d38-983c-ec6e50e20c80-kube-api-access-wfkrl\") pod \"40888f18-1d7d-4d38-983c-ec6e50e20c80\" (UID: \"40888f18-1d7d-4d38-983c-ec6e50e20c80\") "
Sep 09 21:04:10.893146 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:10.893119    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40888f18-1d7d-4d38-983c-ec6e50e20c80-kube-api-access-wfkrl" (OuterVolumeSpecName: "kube-api-access-wfkrl") pod "40888f18-1d7d-4d38-983c-ec6e50e20c80" (UID: "40888f18-1d7d-4d38-983c-ec6e50e20c80"). InnerVolumeSpecName "kube-api-access-wfkrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:04:10.898999 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:10.898977    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7675-a84178"]
Sep 09 21:04:10.902281 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:10.902262    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7675-a84178"]
Sep 09 21:04:10.992607 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:10.992521    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfkrl\" (UniqueName: \"kubernetes.io/projected/40888f18-1d7d-4d38-983c-ec6e50e20c80-kube-api-access-wfkrl\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:04:11.034973 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.034935    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7675-c04a63"]
Sep 09 21:04:11.035325 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.035311    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy"
Sep 09 21:04:11.035385 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.035326    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7f3bc29-b848-4d1b-8fe5-f06925a830de" containerName="istio-proxy"
Sep 09 21:04:11.035385 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.035347    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40888f18-1d7d-4d38-983c-ec6e50e20c80" containerName="mt-curl-7675-a84178"
Sep 09 21:04:11.035385 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.035352    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="40888f18-1d7d-4d38-983c-ec6e50e20c80" containerName="mt-curl-7675-a84178"
Sep 09 21:04:11.035491 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.035409    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="40888f18-1d7d-4d38-983c-ec6e50e20c80" containerName="mt-curl-7675-a84178"
Sep 09 21:04:11.038453 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.038435    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7675-c04a63"
Sep 09 21:04:11.046181 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.046141    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7675-c04a63"]
Sep 09 21:04:11.093943 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.093912    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52pvr\" (UniqueName: \"kubernetes.io/projected/5f2ce745-d3c1-4eae-a447-64b840ad20f5-kube-api-access-52pvr\") pod \"mt-curl-7675-c04a63\" (UID: \"5f2ce745-d3c1-4eae-a447-64b840ad20f5\") " pod="opendatahub/mt-curl-7675-c04a63"
Sep 09 21:04:11.195001 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.194970    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52pvr\" (UniqueName: \"kubernetes.io/projected/5f2ce745-d3c1-4eae-a447-64b840ad20f5-kube-api-access-52pvr\") pod \"mt-curl-7675-c04a63\" (UID: \"5f2ce745-d3c1-4eae-a447-64b840ad20f5\") " pod="opendatahub/mt-curl-7675-c04a63"
Sep 09 21:04:11.202695 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.202666    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52pvr\" (UniqueName: \"kubernetes.io/projected/5f2ce745-d3c1-4eae-a447-64b840ad20f5-kube-api-access-52pvr\") pod \"mt-curl-7675-c04a63\" (UID: \"5f2ce745-d3c1-4eae-a447-64b840ad20f5\") " pod="opendatahub/mt-curl-7675-c04a63"
Sep 09 21:04:11.348982 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.348898    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7675-c04a63"
Sep 09 21:04:11.619070 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.619034    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7675-c04a63"]
Sep 09 21:04:11.620629 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:04:11.620598    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f2ce745_d3c1_4eae_a447_64b840ad20f5.slice/crio-b7694f05398b6deed63a6ed22164383c1b486234f8c3d47b7a2000e8d3842008 WatchSource:0}: Error finding container b7694f05398b6deed63a6ed22164383c1b486234f8c3d47b7a2000e8d3842008: Status 404 returned error can't find the container with id b7694f05398b6deed63a6ed22164383c1b486234f8c3d47b7a2000e8d3842008
Sep 09 21:04:11.725415 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.725387    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7675-a84178"
Sep 09 21:04:11.725556 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.725386    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13c75620c4d3c9adc6414ea895cfd32c856401f674994736c5d9a2e4933ab1b0"
Sep 09 21:04:11.726580 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:11.726554    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7675-c04a63" event={"ID":"5f2ce745-d3c1-4eae-a447-64b840ad20f5","Type":"ContainerStarted","Data":"b7694f05398b6deed63a6ed22164383c1b486234f8c3d47b7a2000e8d3842008"}
Sep 09 21:04:11.736568 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:04:11.736539    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7675-a84178\" is forbidden: User \"system:node:ip-10-0-140-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-126.ec2.internal' and this object" podUID="40888f18-1d7d-4d38-983c-ec6e50e20c80" pod="opendatahub/mt-curl-7675-a84178"
Sep 09 21:04:12.279624 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:12.279587    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40888f18-1d7d-4d38-983c-ec6e50e20c80" path="/var/lib/kubelet/pods/40888f18-1d7d-4d38-983c-ec6e50e20c80/volumes"
Sep 09 21:04:12.734192 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:12.734148    2550 generic.go:356] "Generic (PLEG): container finished" podID="5f2ce745-d3c1-4eae-a447-64b840ad20f5" containerID="55895ada1413f9dc6e0d0761585f6b930a63a8cd538cc3221b768429f5a5fbc4" exitCode=0
Sep 09 21:04:12.734369 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:12.734278    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7675-c04a63" event={"ID":"5f2ce745-d3c1-4eae-a447-64b840ad20f5","Type":"ContainerDied","Data":"55895ada1413f9dc6e0d0761585f6b930a63a8cd538cc3221b768429f5a5fbc4"}
Sep 09 21:04:13.883976 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:13.883938    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7675-c04a63"
Sep 09 21:04:13.905175 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:13.905142    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7675-c04a63_5f2ce745-d3c1-4eae-a447-64b840ad20f5/mt-curl-7675-c04a63/0.log"
Sep 09 21:04:13.918464 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:13.918437    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52pvr\" (UniqueName: \"kubernetes.io/projected/5f2ce745-d3c1-4eae-a447-64b840ad20f5-kube-api-access-52pvr\") pod \"5f2ce745-d3c1-4eae-a447-64b840ad20f5\" (UID: \"5f2ce745-d3c1-4eae-a447-64b840ad20f5\") "
Sep 09 21:04:13.920449 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:13.920420    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f2ce745-d3c1-4eae-a447-64b840ad20f5-kube-api-access-52pvr" (OuterVolumeSpecName: "kube-api-access-52pvr") pod "5f2ce745-d3c1-4eae-a447-64b840ad20f5" (UID: "5f2ce745-d3c1-4eae-a447-64b840ad20f5"). InnerVolumeSpecName "kube-api-access-52pvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:04:13.936800 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:13.936775    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7675-c04a63"]
Sep 09 21:04:13.941084 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:13.941063    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7675-c04a63"]
Sep 09 21:04:14.019377 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:14.019292    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52pvr\" (UniqueName: \"kubernetes.io/projected/5f2ce745-d3c1-4eae-a447-64b840ad20f5-kube-api-access-52pvr\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:04:14.279631 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:14.279554    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f2ce745-d3c1-4eae-a447-64b840ad20f5" path="/var/lib/kubelet/pods/5f2ce745-d3c1-4eae-a447-64b840ad20f5/volumes"
Sep 09 21:04:14.747763 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:14.747736    2550 scope.go:117] "RemoveContainer" containerID="55895ada1413f9dc6e0d0761585f6b930a63a8cd538cc3221b768429f5a5fbc4"
Sep 09 21:04:14.747763 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:14.747752    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7675-c04a63"
Sep 09 21:04:29.331951 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:29.331913    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:30.293087 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:30.293056    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:30.943399 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:30.943370    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"]
Sep 09 21:04:30.965650 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:30.965622    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:31.704721 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:31.704682    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:41.797704 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.797673    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"]
Sep 09 21:04:41.802010 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.798123    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f2ce745-d3c1-4eae-a447-64b840ad20f5" containerName="mt-curl-7675-c04a63"
Sep 09 21:04:41.802010 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.798138    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f2ce745-d3c1-4eae-a447-64b840ad20f5" containerName="mt-curl-7675-c04a63"
Sep 09 21:04:41.802010 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.798232    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f2ce745-d3c1-4eae-a447-64b840ad20f5" containerName="mt-curl-7675-c04a63"
Sep 09 21:04:41.802952 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.802936    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:41.804932 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.804908    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-cb82f2\"/\"kube-root-ca.crt\""
Sep 09 21:04:41.805060 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.804936    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-cb82f2\"/\"openshift-service-ca.crt\""
Sep 09 21:04:41.805389 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.805372    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-cb82f2\"/\"default-dockercfg-bjtcp\""
Sep 09 21:04:41.805452 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.805404    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-cb82f2\"/\"rate-test-model-cb82f2-kserve-self-signed-certs\""
Sep 09 21:04:41.810195 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.810157    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"]
Sep 09 21:04:41.958557 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.958528    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-tmp-dir\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:41.958750 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.958562    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6492bd-ac38-497f-832e-7b44fba11aa8-tls-certs\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:41.958750 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.958589    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k29qt\" (UniqueName: \"kubernetes.io/projected/ba6492bd-ac38-497f-832e-7b44fba11aa8-kube-api-access-k29qt\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:41.958750 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.958690    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-home\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:41.958750 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.958725    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-model-cache\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:41.958950 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:41.958786    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-dshm\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.060111 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.060020    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-tmp-dir\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.060111 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.060056    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6492bd-ac38-497f-832e-7b44fba11aa8-tls-certs\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.060111 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.060074    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k29qt\" (UniqueName: \"kubernetes.io/projected/ba6492bd-ac38-497f-832e-7b44fba11aa8-kube-api-access-k29qt\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.060415 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.060205    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-home\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.060415 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.060250    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-model-cache\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.060415 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.060293    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-dshm\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.060572 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.060490    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-tmp-dir\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.060572 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.060525    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-home\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.060653 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.060611    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-model-cache\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.062422 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.062399    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-dshm\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.062647 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.062629    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6492bd-ac38-497f-832e-7b44fba11aa8-tls-certs\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.070123 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.070104    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k29qt\" (UniqueName: \"kubernetes.io/projected/ba6492bd-ac38-497f-832e-7b44fba11aa8-kube-api-access-k29qt\") pod \"rate-test-model-cb82f2-kserve-565c85b78c-2hvrg\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") " pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.114841 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.114820    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:42.140742 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.140712    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:42.391551 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.391529    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"]
Sep 09 21:04:42.393044 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:04:42.393019    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba6492bd_ac38_497f_832e_7b44fba11aa8.slice/crio-5d5e2bcd13b5787735a12ff5de2b62105908b13fdf6c3e0caf02e5deb6a70d26 WatchSource:0}: Error finding container 5d5e2bcd13b5787735a12ff5de2b62105908b13fdf6c3e0caf02e5deb6a70d26: Status 404 returned error can't find the container with id 5d5e2bcd13b5787735a12ff5de2b62105908b13fdf6c3e0caf02e5deb6a70d26
Sep 09 21:04:42.907460 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.907421    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg" event={"ID":"ba6492bd-ac38-497f-832e-7b44fba11aa8","Type":"ContainerStarted","Data":"b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c"}
Sep 09 21:04:42.907893 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.907468    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg" event={"ID":"ba6492bd-ac38-497f-832e-7b44fba11aa8","Type":"ContainerStarted","Data":"5d5e2bcd13b5787735a12ff5de2b62105908b13fdf6c3e0caf02e5deb6a70d26"}
Sep 09 21:04:42.924354 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:42.924311    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg" podStartSLOduration=1.924297404 podStartE2EDuration="1.924297404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:42.923540513 +0000 UTC m=+1943.074212867" watchObservedRunningTime="2026-09-09 21:04:42.924297404 +0000 UTC m=+1943.074969780"
Sep 09 21:04:45.966411 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:45.966349    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg" podUID="1529e5ef-f5e4-409d-b5cd-630e089bedca" containerName="main" containerID="cri-o://8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f" gracePeriod=45
Sep 09 21:04:46.228153 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.228080    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:04:46.398512 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.398473    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-dshm\") pod \"1529e5ef-f5e4-409d-b5cd-630e089bedca\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") "
Sep 09 21:04:46.398677 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.398541    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-model-cache\") pod \"1529e5ef-f5e4-409d-b5cd-630e089bedca\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") "
Sep 09 21:04:46.398677 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.398598    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1529e5ef-f5e4-409d-b5cd-630e089bedca-tls-certs\") pod \"1529e5ef-f5e4-409d-b5cd-630e089bedca\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") "
Sep 09 21:04:46.398677 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.398635    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-tmp-dir\") pod \"1529e5ef-f5e4-409d-b5cd-630e089bedca\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") "
Sep 09 21:04:46.398677 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.398673    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-home\") pod \"1529e5ef-f5e4-409d-b5cd-630e089bedca\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") "
Sep 09 21:04:46.398872 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.398751    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgdqj\" (UniqueName: \"kubernetes.io/projected/1529e5ef-f5e4-409d-b5cd-630e089bedca-kube-api-access-pgdqj\") pod \"1529e5ef-f5e4-409d-b5cd-630e089bedca\" (UID: \"1529e5ef-f5e4-409d-b5cd-630e089bedca\") "
Sep 09 21:04:46.398872 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.398760    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-model-cache" (OuterVolumeSpecName: "model-cache") pod "1529e5ef-f5e4-409d-b5cd-630e089bedca" (UID: "1529e5ef-f5e4-409d-b5cd-630e089bedca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:04:46.398984 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.398893    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1529e5ef-f5e4-409d-b5cd-630e089bedca" (UID: "1529e5ef-f5e4-409d-b5cd-630e089bedca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:04:46.398984 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.398966    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-home" (OuterVolumeSpecName: "home") pod "1529e5ef-f5e4-409d-b5cd-630e089bedca" (UID: "1529e5ef-f5e4-409d-b5cd-630e089bedca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:04:46.399136 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.399115    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:04:46.399219 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.399143    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:04:46.399219 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.399156    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:04:46.401002 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.400972    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1529e5ef-f5e4-409d-b5cd-630e089bedca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1529e5ef-f5e4-409d-b5cd-630e089bedca" (UID: "1529e5ef-f5e4-409d-b5cd-630e089bedca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:04:46.401132 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.401077    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-dshm" (OuterVolumeSpecName: "dshm") pod "1529e5ef-f5e4-409d-b5cd-630e089bedca" (UID: "1529e5ef-f5e4-409d-b5cd-630e089bedca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:04:46.401229 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.401123    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1529e5ef-f5e4-409d-b5cd-630e089bedca-kube-api-access-pgdqj" (OuterVolumeSpecName: "kube-api-access-pgdqj") pod "1529e5ef-f5e4-409d-b5cd-630e089bedca" (UID: "1529e5ef-f5e4-409d-b5cd-630e089bedca"). InnerVolumeSpecName "kube-api-access-pgdqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:04:46.499742 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.499658    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgdqj\" (UniqueName: \"kubernetes.io/projected/1529e5ef-f5e4-409d-b5cd-630e089bedca-kube-api-access-pgdqj\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:04:46.499742 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.499688    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1529e5ef-f5e4-409d-b5cd-630e089bedca-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:04:46.499742 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.499726    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1529e5ef-f5e4-409d-b5cd-630e089bedca-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:04:46.931535 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.931441    2550 generic.go:356] "Generic (PLEG): container finished" podID="1529e5ef-f5e4-409d-b5cd-630e089bedca" containerID="8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f" exitCode=0
Sep 09 21:04:46.931535 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.931493    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"
Sep 09 21:04:46.931535 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.931522    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg" event={"ID":"1529e5ef-f5e4-409d-b5cd-630e089bedca","Type":"ContainerDied","Data":"8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f"}
Sep 09 21:04:46.931822 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.931578    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg" event={"ID":"1529e5ef-f5e4-409d-b5cd-630e089bedca","Type":"ContainerDied","Data":"73c0bdc77fbca49d46d5bd2c3863095bf536353bd3f6ca6cbded8a89b9d395fa"}
Sep 09 21:04:46.931822 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.931597    2550 scope.go:117] "RemoveContainer" containerID="8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f"
Sep 09 21:04:46.942441 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.942426    2550 scope.go:117] "RemoveContainer" containerID="8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f"
Sep 09 21:04:46.942740 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:04:46.942721    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f\": container with ID starting with 8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f not found: ID does not exist" containerID="8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f"
Sep 09 21:04:46.942803 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.942748    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f"} err="failed to get container status \"8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f\": rpc error: code = NotFound desc = could not find container \"8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f\": container with ID starting with 8fb820162924e05cf6a259a78a626b350ad8cc20c40bcb1d8cd8f3c1c48afc2f not found: ID does not exist"
Sep 09 21:04:46.955066 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.955045    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"]
Sep 09 21:04:46.959175 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:46.959148    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/sub-test-model-a2fb5a-kserve-7995fb9fb9-7hxtg"]
Sep 09 21:04:48.279482 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:48.279436    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1529e5ef-f5e4-409d-b5cd-630e089bedca" path="/var/lib/kubelet/pods/1529e5ef-f5e4-409d-b5cd-630e089bedca/volumes"
Sep 09 21:04:50.558574 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:50.558501    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:52.115550 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:52.115514    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:52.115550 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:52.115561    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:52.128328 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:52.128300    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:04:52.976794 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:04:52.976764    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:05:07.003094 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.003053    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"]
Sep 09 21:05:07.003502 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.003468    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1529e5ef-f5e4-409d-b5cd-630e089bedca" containerName="main"
Sep 09 21:05:07.003502 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.003482    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1529e5ef-f5e4-409d-b5cd-630e089bedca" containerName="main"
Sep 09 21:05:07.003628 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.003569    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1529e5ef-f5e4-409d-b5cd-630e089bedca" containerName="main"
Sep 09 21:05:07.006543 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.006525    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.009413 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.009388    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-a2fb5a\"/\"default-dockercfg-7v86l\""
Sep 09 21:05:07.009413 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.009406    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-a2fb5a\"/\"rate-test-model-a2fb5a-kserve-self-signed-certs\""
Sep 09 21:05:07.009575 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.009390    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-a2fb5a\"/\"openshift-service-ca.crt\""
Sep 09 21:05:07.009575 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.009390    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-a2fb5a\"/\"kube-root-ca.crt\""
Sep 09 21:05:07.018695 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.018674    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"]
Sep 09 21:05:07.070461 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.070428    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-home\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.070663 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.070467    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-tmp-dir\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.070663 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.070605    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebd056b-a28c-4a23-b493-6054b1deed24-tls-certs\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.070663 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.070639    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntqhw\" (UniqueName: \"kubernetes.io/projected/3ebd056b-a28c-4a23-b493-6054b1deed24-kube-api-access-ntqhw\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.070841 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.070716    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-model-cache\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.070841 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.070740    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-dshm\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.086602 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.086571    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:07.171639 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.171606    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebd056b-a28c-4a23-b493-6054b1deed24-tls-certs\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.171840 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.171656    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntqhw\" (UniqueName: \"kubernetes.io/projected/3ebd056b-a28c-4a23-b493-6054b1deed24-kube-api-access-ntqhw\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.171840 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.171706    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-model-cache\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.171840 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.171731    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-dshm\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.171840 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.171756    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-home\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.171840 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.171789    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-tmp-dir\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.172213 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.172191    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-home\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.172306 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.172224    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-model-cache\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.172306 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.172249    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-tmp-dir\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.174136 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.174112    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-dshm\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.174244 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.174140    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebd056b-a28c-4a23-b493-6054b1deed24-tls-certs\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.180141 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.180119    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntqhw\" (UniqueName: \"kubernetes.io/projected/3ebd056b-a28c-4a23-b493-6054b1deed24-kube-api-access-ntqhw\") pod \"rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") " pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.315415 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.315325    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:07.595123 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:07.595086    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"]
Sep 09 21:05:07.596606 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:05:07.596582    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ebd056b_a28c_4a23_b493_6054b1deed24.slice/crio-b8128ad188123c483ab1b5f9a2d0698798a453b35b85b92190c47c912ad8a847 WatchSource:0}: Error finding container b8128ad188123c483ab1b5f9a2d0698798a453b35b85b92190c47c912ad8a847: Status 404 returned error can't find the container with id b8128ad188123c483ab1b5f9a2d0698798a453b35b85b92190c47c912ad8a847
Sep 09 21:05:08.057274 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:08.057241    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6" event={"ID":"3ebd056b-a28c-4a23-b493-6054b1deed24","Type":"ContainerStarted","Data":"bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec"}
Sep 09 21:05:08.057274 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:08.057288    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6" event={"ID":"3ebd056b-a28c-4a23-b493-6054b1deed24","Type":"ContainerStarted","Data":"b8128ad188123c483ab1b5f9a2d0698798a453b35b85b92190c47c912ad8a847"}
Sep 09 21:05:08.073766 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:08.073709    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6" podStartSLOduration=2.073691496 podStartE2EDuration="2.073691496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:05:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:05:08.071902478 +0000 UTC m=+1968.222574820" watchObservedRunningTime="2026-09-09 21:05:08.073691496 +0000 UTC m=+1968.224363860"
Sep 09 21:05:09.124443 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:09.124404    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:09.192331 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:09.192302    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:10.645677 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:10.645647    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:10.708519 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:10.708483    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:13.818683 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:13.818599    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:17.315465 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:17.315428    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:17.315939 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:17.315520    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:17.328180 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:17.328138    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:18.123122 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:18.123091    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:05:20.720547 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:20.720518    2550 scope.go:117] "RemoveContainer" containerID="785e17b496f745837f48b926ef1cab1a26a8e038dd27d6240e2f93f529669626"
Sep 09 21:05:41.936751 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:41.936709    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:44.494503 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:44.494467    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"]
Sep 09 21:05:44.516264 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:44.516237    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:45.571565 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:45.571536    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"]
Sep 09 21:05:45.591819 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:45.591791    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:46.009552 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.009523    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:46.267061 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.266990    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"]
Sep 09 21:05:46.270629 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.270611    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.272744 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.272722    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-35e9ca-openshift-default-dockercfg-lljx2\""
Sep 09 21:05:46.282652 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.282633    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"]
Sep 09 21:05:46.289576 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.289547    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:46.416126 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.416097    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-envoy\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.416306 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.416142    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-token\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.416306 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.416271    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istiod-ca-cert\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.416399 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.416327    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-socket\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.416671 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.416647    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-certs\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.416789 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.416737    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-credential-socket\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.416789 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.416770    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-podinfo\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.416895 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.416797    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-data\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.416895 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.416867    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m79lx\" (UniqueName: \"kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-kube-api-access-m79lx\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.517610 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.517534    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-credential-socket\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.517610 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.517566    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-podinfo\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.517610 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.517583    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-data\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.517610 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.517603    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m79lx\" (UniqueName: \"kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-kube-api-access-m79lx\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.517901 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.517635    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-envoy\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.517901 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.517652    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-token\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.517901 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.517690    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istiod-ca-cert\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.517901 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.517802    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-socket\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.517901 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.517863    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-certs\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.518138 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.517969    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-data\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.518249 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.518229    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-certs\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.518317 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.518257    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-socket\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.518405 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.518386    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-credential-socket\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.518492 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.518476    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istiod-ca-cert\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.519883 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.519858    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-envoy\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.520105 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.520085    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-podinfo\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.525729 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.525710    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m79lx\" (UniqueName: \"kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-kube-api-access-m79lx\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.525839 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.525818    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-token\") pod \"e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") " pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.582114 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.582093    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:46.860884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.860852    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"]
Sep 09 21:05:46.862762 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:05:46.862737    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eaae6c9_9519_4924_8c3b_a60e5bcaa79e.slice/crio-f1b924bf3d23a9a5b614ffee97928fa73a52bf22c1ad4debb7362af91a35a0bc WatchSource:0}: Error finding container f1b924bf3d23a9a5b614ffee97928fa73a52bf22c1ad4debb7362af91a35a0bc: Status 404 returned error can't find the container with id f1b924bf3d23a9a5b614ffee97928fa73a52bf22c1ad4debb7362af91a35a0bc
Sep 09 21:05:46.864981 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.864948    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:05:46.865095 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.865013    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:05:46.865095 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.865043    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:05:46.972472 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:46.972435    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:47.270568 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:47.270530    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" event={"ID":"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e","Type":"ContainerStarted","Data":"67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e"}
Sep 09 21:05:47.270733 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:47.270590    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" event={"ID":"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e","Type":"ContainerStarted","Data":"f1b924bf3d23a9a5b614ffee97928fa73a52bf22c1ad4debb7362af91a35a0bc"}
Sep 09 21:05:47.291679 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:47.291624    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" podStartSLOduration=1.291603861 podStartE2EDuration="1.291603861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:05:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:05:47.289577202 +0000 UTC m=+2007.440249546" watchObservedRunningTime="2026-09-09 21:05:47.291603861 +0000 UTC m=+2007.442276204"
Sep 09 21:05:47.582538 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:47.582462    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:47.587679 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:47.587657    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:48.279203 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:48.279152    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:48.279436 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:48.279243    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:05:49.167394 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:49.167359    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:49.269670 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:49.269623    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:50.350521 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:50.350487    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:50.455673 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:50.455633    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:50.734546 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:50.734513    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:50.940554 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:50.940516    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:51.040487 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.040390    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:51.093741 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.093709    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"]
Sep 09 21:05:51.097329 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.097314    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:05:51.099602 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.099577    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-35e9ca\""
Sep 09 21:05:51.100049 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.100036    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-35e9ca-dockercfg-46k4n\""
Sep 09 21:05:51.108455 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.108432    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"]
Sep 09 21:05:51.161528 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.161488    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/048a5687-42a4-4b44-beee-db5bd2192a1f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8\" (UID: \"048a5687-42a4-4b44-beee-db5bd2192a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:05:51.161729 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.161597    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6bzd\" (UniqueName: \"kubernetes.io/projected/048a5687-42a4-4b44-beee-db5bd2192a1f-kube-api-access-p6bzd\") pod \"payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8\" (UID: \"048a5687-42a4-4b44-beee-db5bd2192a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:05:51.262554 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.262517    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/048a5687-42a4-4b44-beee-db5bd2192a1f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8\" (UID: \"048a5687-42a4-4b44-beee-db5bd2192a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:05:51.262733 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.262679    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6bzd\" (UniqueName: \"kubernetes.io/projected/048a5687-42a4-4b44-beee-db5bd2192a1f-kube-api-access-p6bzd\") pod \"payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8\" (UID: \"048a5687-42a4-4b44-beee-db5bd2192a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:05:51.263230 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.263211    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/048a5687-42a4-4b44-beee-db5bd2192a1f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8\" (UID: \"048a5687-42a4-4b44-beee-db5bd2192a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:05:51.272253 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.272235    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6bzd\" (UniqueName: \"kubernetes.io/projected/048a5687-42a4-4b44-beee-db5bd2192a1f-kube-api-access-p6bzd\") pod \"payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8\" (UID: \"048a5687-42a4-4b44-beee-db5bd2192a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:05:51.410249 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.410136    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:05:51.682186 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:51.682148    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"]
Sep 09 21:05:51.683555 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:05:51.683518    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048a5687_42a4_4b44_beee_db5bd2192a1f.slice/crio-a23d58a36cca14e8adc8cb2783baa16abd58c39ec974bf0c229d869e7f4067b1 WatchSource:0}: Error finding container a23d58a36cca14e8adc8cb2783baa16abd58c39ec974bf0c229d869e7f4067b1: Status 404 returned error can't find the container with id a23d58a36cca14e8adc8cb2783baa16abd58c39ec974bf0c229d869e7f4067b1
Sep 09 21:05:52.301661 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:52.301629    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8" event={"ID":"048a5687-42a4-4b44-beee-db5bd2192a1f","Type":"ContainerStarted","Data":"f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa"}
Sep 09 21:05:52.301835 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:52.301688    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:05:52.301835 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:52.301711    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8" event={"ID":"048a5687-42a4-4b44-beee-db5bd2192a1f","Type":"ContainerStarted","Data":"a23d58a36cca14e8adc8cb2783baa16abd58c39ec974bf0c229d869e7f4067b1"}
Sep 09 21:05:52.318800 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:52.318742    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8" podStartSLOduration=1.31872638 podStartE2EDuration="1.31872638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:05:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:05:52.316725747 +0000 UTC m=+2012.467398090" watchObservedRunningTime="2026-09-09 21:05:52.31872638 +0000 UTC m=+2012.469398723"
Sep 09 21:05:59.514192 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.514105    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg" podUID="ba6492bd-ac38-497f-832e-7b44fba11aa8" containerName="main" containerID="cri-o://b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c" gracePeriod=45
Sep 09 21:05:59.811236 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.811208    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:05:59.862294 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.862248    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-model-cache\") pod \"ba6492bd-ac38-497f-832e-7b44fba11aa8\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") "
Sep 09 21:05:59.862477 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.862361    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k29qt\" (UniqueName: \"kubernetes.io/projected/ba6492bd-ac38-497f-832e-7b44fba11aa8-kube-api-access-k29qt\") pod \"ba6492bd-ac38-497f-832e-7b44fba11aa8\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") "
Sep 09 21:05:59.862477 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.862413    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-tmp-dir\") pod \"ba6492bd-ac38-497f-832e-7b44fba11aa8\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") "
Sep 09 21:05:59.862477 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.862443    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6492bd-ac38-497f-832e-7b44fba11aa8-tls-certs\") pod \"ba6492bd-ac38-497f-832e-7b44fba11aa8\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") "
Sep 09 21:05:59.862668 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.862491    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-home\") pod \"ba6492bd-ac38-497f-832e-7b44fba11aa8\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") "
Sep 09 21:05:59.862668 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.862531    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-dshm\") pod \"ba6492bd-ac38-497f-832e-7b44fba11aa8\" (UID: \"ba6492bd-ac38-497f-832e-7b44fba11aa8\") "
Sep 09 21:05:59.862778 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.862666    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba6492bd-ac38-497f-832e-7b44fba11aa8" (UID: "ba6492bd-ac38-497f-832e-7b44fba11aa8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:05:59.862887 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.862858    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-home" (OuterVolumeSpecName: "home") pod "ba6492bd-ac38-497f-832e-7b44fba11aa8" (UID: "ba6492bd-ac38-497f-832e-7b44fba11aa8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:05:59.862947 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.862902    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:05:59.863008 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.862962    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba6492bd-ac38-497f-832e-7b44fba11aa8" (UID: "ba6492bd-ac38-497f-832e-7b44fba11aa8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:05:59.864742 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.864712    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba6492bd-ac38-497f-832e-7b44fba11aa8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba6492bd-ac38-497f-832e-7b44fba11aa8" (UID: "ba6492bd-ac38-497f-832e-7b44fba11aa8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:05:59.864920 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.864903    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba6492bd-ac38-497f-832e-7b44fba11aa8-kube-api-access-k29qt" (OuterVolumeSpecName: "kube-api-access-k29qt") pod "ba6492bd-ac38-497f-832e-7b44fba11aa8" (UID: "ba6492bd-ac38-497f-832e-7b44fba11aa8"). InnerVolumeSpecName "kube-api-access-k29qt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:05:59.865001 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.864913    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-dshm" (OuterVolumeSpecName: "dshm") pod "ba6492bd-ac38-497f-832e-7b44fba11aa8" (UID: "ba6492bd-ac38-497f-832e-7b44fba11aa8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:05:59.963976 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.963937    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:05:59.963976 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.963966    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k29qt\" (UniqueName: \"kubernetes.io/projected/ba6492bd-ac38-497f-832e-7b44fba11aa8-kube-api-access-k29qt\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:05:59.963976 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.963977    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6492bd-ac38-497f-832e-7b44fba11aa8-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:05:59.963976 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.963985    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:05:59.964255 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:05:59.963992    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6492bd-ac38-497f-832e-7b44fba11aa8-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:06:00.351248 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.351210    2550 generic.go:356] "Generic (PLEG): container finished" podID="ba6492bd-ac38-497f-832e-7b44fba11aa8" containerID="b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c" exitCode=0
Sep 09 21:06:00.351433 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.351282    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg" event={"ID":"ba6492bd-ac38-497f-832e-7b44fba11aa8","Type":"ContainerDied","Data":"b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c"}
Sep 09 21:06:00.351433 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.351340    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg" event={"ID":"ba6492bd-ac38-497f-832e-7b44fba11aa8","Type":"ContainerDied","Data":"5d5e2bcd13b5787735a12ff5de2b62105908b13fdf6c3e0caf02e5deb6a70d26"}
Sep 09 21:06:00.351433 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.351362    2550 scope.go:117] "RemoveContainer" containerID="b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c"
Sep 09 21:06:00.351668 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.351650    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"
Sep 09 21:06:00.366467 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.366446    2550 scope.go:117] "RemoveContainer" containerID="b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c"
Sep 09 21:06:00.366774 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:06:00.366752    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c\": container with ID starting with b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c not found: ID does not exist" containerID="b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c"
Sep 09 21:06:00.366841 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.366785    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c"} err="failed to get container status \"b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c\": rpc error: code = NotFound desc = could not find container \"b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c\": container with ID starting with b5c7a818ec804f790a1db58e2f3a59d00de4b397cb829a9afb45d7ad895cb95c not found: ID does not exist"
Sep 09 21:06:00.368497 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.368477    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"]
Sep 09 21:06:00.370636 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.370616    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cb82f2/rate-test-model-cb82f2-kserve-565c85b78c-2hvrg"]
Sep 09 21:06:00.591122 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.591033    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6" podUID="3ebd056b-a28c-4a23-b493-6054b1deed24" containerName="main" containerID="cri-o://bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec" gracePeriod=45
Sep 09 21:06:00.875743 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.875720    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:06:00.974297 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.974271    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-dshm\") pod \"3ebd056b-a28c-4a23-b493-6054b1deed24\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") "
Sep 09 21:06:00.974434 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.974319    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntqhw\" (UniqueName: \"kubernetes.io/projected/3ebd056b-a28c-4a23-b493-6054b1deed24-kube-api-access-ntqhw\") pod \"3ebd056b-a28c-4a23-b493-6054b1deed24\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") "
Sep 09 21:06:00.974434 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.974339    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-model-cache\") pod \"3ebd056b-a28c-4a23-b493-6054b1deed24\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") "
Sep 09 21:06:00.974434 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.974375    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebd056b-a28c-4a23-b493-6054b1deed24-tls-certs\") pod \"3ebd056b-a28c-4a23-b493-6054b1deed24\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") "
Sep 09 21:06:00.974434 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.974416    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-home\") pod \"3ebd056b-a28c-4a23-b493-6054b1deed24\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") "
Sep 09 21:06:00.974649 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.974485    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-tmp-dir\") pod \"3ebd056b-a28c-4a23-b493-6054b1deed24\" (UID: \"3ebd056b-a28c-4a23-b493-6054b1deed24\") "
Sep 09 21:06:00.974706 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.974657    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-model-cache" (OuterVolumeSpecName: "model-cache") pod "3ebd056b-a28c-4a23-b493-6054b1deed24" (UID: "3ebd056b-a28c-4a23-b493-6054b1deed24"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:06:00.974776 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.974732    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-home" (OuterVolumeSpecName: "home") pod "3ebd056b-a28c-4a23-b493-6054b1deed24" (UID: "3ebd056b-a28c-4a23-b493-6054b1deed24"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:06:00.974838 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.974810    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3ebd056b-a28c-4a23-b493-6054b1deed24" (UID: "3ebd056b-a28c-4a23-b493-6054b1deed24"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:06:00.976474 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.976437    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:06:00.976474 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.976465    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:06:00.976474 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.976478    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:06:00.976679 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.976543    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-dshm" (OuterVolumeSpecName: "dshm") pod "3ebd056b-a28c-4a23-b493-6054b1deed24" (UID: "3ebd056b-a28c-4a23-b493-6054b1deed24"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:06:00.976904 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.976872    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ebd056b-a28c-4a23-b493-6054b1deed24-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3ebd056b-a28c-4a23-b493-6054b1deed24" (UID: "3ebd056b-a28c-4a23-b493-6054b1deed24"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:06:00.976904 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:00.976890    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ebd056b-a28c-4a23-b493-6054b1deed24-kube-api-access-ntqhw" (OuterVolumeSpecName: "kube-api-access-ntqhw") pod "3ebd056b-a28c-4a23-b493-6054b1deed24" (UID: "3ebd056b-a28c-4a23-b493-6054b1deed24"). InnerVolumeSpecName "kube-api-access-ntqhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:06:01.076858 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.076829    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ebd056b-a28c-4a23-b493-6054b1deed24-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:06:01.076858 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.076855    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntqhw\" (UniqueName: \"kubernetes.io/projected/3ebd056b-a28c-4a23-b493-6054b1deed24-kube-api-access-ntqhw\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:06:01.076858 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.076866    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebd056b-a28c-4a23-b493-6054b1deed24-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:06:01.359001 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.358969    2550 generic.go:356] "Generic (PLEG): container finished" podID="3ebd056b-a28c-4a23-b493-6054b1deed24" containerID="bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec" exitCode=0
Sep 09 21:06:01.359191 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.359010    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"
Sep 09 21:06:01.359191 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.359056    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6" event={"ID":"3ebd056b-a28c-4a23-b493-6054b1deed24","Type":"ContainerDied","Data":"bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec"}
Sep 09 21:06:01.359191 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.359100    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6" event={"ID":"3ebd056b-a28c-4a23-b493-6054b1deed24","Type":"ContainerDied","Data":"b8128ad188123c483ab1b5f9a2d0698798a453b35b85b92190c47c912ad8a847"}
Sep 09 21:06:01.359191 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.359122    2550 scope.go:117] "RemoveContainer" containerID="bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec"
Sep 09 21:06:01.369664 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.369642    2550 scope.go:117] "RemoveContainer" containerID="bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec"
Sep 09 21:06:01.370120 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:06:01.370055    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec\": container with ID starting with bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec not found: ID does not exist" containerID="bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec"
Sep 09 21:06:01.370120 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.370089    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec"} err="failed to get container status \"bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec\": rpc error: code = NotFound desc = could not find container \"bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec\": container with ID starting with bf47364ba6b865cd5169afcad1fbcb530536f2dfce34f263122035753fb7fbec not found: ID does not exist"
Sep 09 21:06:01.383305 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.383281    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"]
Sep 09 21:06:01.388499 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:01.388474    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a2fb5a/rate-test-model-a2fb5a-kserve-85f6dcb647-m8hh6"]
Sep 09 21:06:02.279688 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:02.279646    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ebd056b-a28c-4a23-b493-6054b1deed24" path="/var/lib/kubelet/pods/3ebd056b-a28c-4a23-b493-6054b1deed24/volumes"
Sep 09 21:06:02.280148 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:02.280044    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba6492bd-ac38-497f-832e-7b44fba11aa8" path="/var/lib/kubelet/pods/ba6492bd-ac38-497f-832e-7b44fba11aa8/volumes"
Sep 09 21:06:03.309101 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:03.309063    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:06:03.728682 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:03.728504    2550 patch_prober.go:28] interesting pod/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": read tcp 10.132.0.2:39032->10.132.0.95:15021: read: connection reset by peer" start-of-body=
Sep 09 21:06:03.728682 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:03.728581    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": read tcp 10.132.0.2:39032->10.132.0.95:15021: read: connection reset by peer"
Sep 09 21:06:04.381833 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:04.381801    2550 generic.go:356] "Generic (PLEG): container finished" podID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerID="67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e" exitCode=0
Sep 09 21:06:04.382256 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:04.381850    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" event={"ID":"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e","Type":"ContainerDied","Data":"67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e"}
Sep 09 21:06:04.382307 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:04.382257    2550 scope.go:117] "RemoveContainer" containerID="67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e"
Sep 09 21:06:04.383301 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:04.383271    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:04.383410 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:04.383352    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:04.383410 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:04.383399    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:05.393311 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:05.393279    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" event={"ID":"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e","Type":"ContainerStarted","Data":"3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493"}
Sep 09 21:06:05.582951 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:05.582920    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:06:05.583127 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:05.583052    2550 patch_prober.go:28] interesting pod/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 09 21:06:05.583127 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:05.583102    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 09 21:06:06.582808 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:06.582768    2550 patch_prober.go:28] interesting pod/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 09 21:06:06.583311 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:06.582835    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 09 21:06:07.583372 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:07.583335    2550 patch_prober.go:28] interesting pod/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 09 21:06:07.583756 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:07.583394    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 09 21:06:08.583104 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:08.583067    2550 patch_prober.go:28] interesting pod/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 09 21:06:08.583306 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:08.583123    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 09 21:06:09.583481 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:09.583440    2550 patch_prober.go:28] interesting pod/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 09 21:06:09.583835 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:09.583513    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 09 21:06:10.582733 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:10.582697    2550 patch_prober.go:28] interesting pod/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 09 21:06:10.582911 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:10.582750    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 09 21:06:11.585948 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:11.585917    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:06:11.586446 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:11.586324    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:06:11.586886 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:11.586866    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:06:14.133724 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:14.133691    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:14.777309 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:14.777276    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:15.068563 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.068486    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"]
Sep 09 21:06:15.068871 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.068859    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba6492bd-ac38-497f-832e-7b44fba11aa8" containerName="main"
Sep 09 21:06:15.068914 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.068873    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba6492bd-ac38-497f-832e-7b44fba11aa8" containerName="main"
Sep 09 21:06:15.068914 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.068886    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ebd056b-a28c-4a23-b493-6054b1deed24" containerName="main"
Sep 09 21:06:15.068914 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.068892    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ebd056b-a28c-4a23-b493-6054b1deed24" containerName="main"
Sep 09 21:06:15.069012 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.068956    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba6492bd-ac38-497f-832e-7b44fba11aa8" containerName="main"
Sep 09 21:06:15.069012 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.068964    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ebd056b-a28c-4a23-b493-6054b1deed24" containerName="main"
Sep 09 21:06:15.071017 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.070993    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.073601 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.073579    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-2ff038-openshift-default-dockercfg-bhpc6\""
Sep 09 21:06:15.085096 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.085070    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"]
Sep 09 21:06:15.108560 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.108529    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:15.212237 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.212203    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-data\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.212237 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.212239    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-credential-socket\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.212708 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.212265    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-envoy\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.212708 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.212292    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bfj8\" (UniqueName: \"kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-kube-api-access-6bfj8\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.212708 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.212334    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3530dd8c-0f36-4309-bdda-5b481c304d02-istiod-ca-cert\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.212708 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.212422    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-socket\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.212708 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.212453    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-token\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.212708 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.212479    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-certs\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.212708 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.212533    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-podinfo\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.313570 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.313511    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-envoy\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.313570 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.313571    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bfj8\" (UniqueName: \"kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-kube-api-access-6bfj8\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.313845 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.313599    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3530dd8c-0f36-4309-bdda-5b481c304d02-istiod-ca-cert\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.313845 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.313650    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-socket\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.313845 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.313666    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-token\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.313845 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.313694    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-certs\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.313845 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.313731    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-podinfo\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.313845 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.313769    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-data\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.313845 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.313795    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-credential-socket\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.314153 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.314072    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-socket\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.314290 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.314265    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-certs\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.314358 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.314301    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-credential-socket\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.314453 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.314429    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-data\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.314512 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.314459    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3530dd8c-0f36-4309-bdda-5b481c304d02-istiod-ca-cert\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.315895 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.315875    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-envoy\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.316128 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.316110    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-podinfo\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.325221 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.322503    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-token\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.325221 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.322587    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bfj8\" (UniqueName: \"kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-kube-api-access-6bfj8\") pod \"e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") " pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.381809 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.381772    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:15.553731 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.553693    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:15.661002 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.660979    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"]
Sep 09 21:06:15.663599 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:06:15.663576    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3530dd8c_0f36_4309_bdda_5b481c304d02.slice/crio-5f6af0166bec00ec08d39dc7747a888c6580aef36d4bc626de372a267de3c170 WatchSource:0}: Error finding container 5f6af0166bec00ec08d39dc7747a888c6580aef36d4bc626de372a267de3c170: Status 404 returned error can't find the container with id 5f6af0166bec00ec08d39dc7747a888c6580aef36d4bc626de372a267de3c170
Sep 09 21:06:15.666250 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.666218    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:15.666319 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.666284    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:15.666319 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:15.666313    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:16.459555 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:16.459517    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" event={"ID":"3530dd8c-0f36-4309-bdda-5b481c304d02","Type":"ContainerStarted","Data":"052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779"}
Sep 09 21:06:16.459930 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:16.459579    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" event={"ID":"3530dd8c-0f36-4309-bdda-5b481c304d02","Type":"ContainerStarted","Data":"5f6af0166bec00ec08d39dc7747a888c6580aef36d4bc626de372a267de3c170"}
Sep 09 21:06:16.480362 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:16.480315    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" podStartSLOduration=1.480300051 podStartE2EDuration="1.480300051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:16.478224982 +0000 UTC m=+2036.628897327" watchObservedRunningTime="2026-09-09 21:06:16.480300051 +0000 UTC m=+2036.630972393"
Sep 09 21:06:17.382567 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:17.382535    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:17.387653 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:17.387628    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:17.465482 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:17.465453    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:17.466386 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:17.466367    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:18.461344 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:18.461309    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:18.545058 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:18.545020    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:19.636116 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:19.636081    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:19.708420 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:19.708389    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:19.837449 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:19.837414    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:20.174814 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.174779    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:20.274382 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.274324    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:20.332116 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.332086    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"]
Sep 09 21:06:20.335196 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.335178    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:06:20.340449 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.340427    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-2ff038-dockercfg-59qtw\""
Sep 09 21:06:20.340578 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.340471    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-2ff038\""
Sep 09 21:06:20.355731 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.355705    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"]
Sep 09 21:06:20.458011 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.457928    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f88pw\" (UniqueName: \"kubernetes.io/projected/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-kube-api-access-f88pw\") pod \"payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv\" (UID: \"4e1fc10d-f0a0-4142-af35-8e86a9e1e037\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:06:20.458011 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.457976    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv\" (UID: \"4e1fc10d-f0a0-4142-af35-8e86a9e1e037\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:06:20.559107 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.559070    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f88pw\" (UniqueName: \"kubernetes.io/projected/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-kube-api-access-f88pw\") pod \"payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv\" (UID: \"4e1fc10d-f0a0-4142-af35-8e86a9e1e037\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:06:20.559107 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.559112    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv\" (UID: \"4e1fc10d-f0a0-4142-af35-8e86a9e1e037\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:06:20.559819 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.559800    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv\" (UID: \"4e1fc10d-f0a0-4142-af35-8e86a9e1e037\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:06:20.570442 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.570420    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f88pw\" (UniqueName: \"kubernetes.io/projected/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-kube-api-access-f88pw\") pod \"payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv\" (UID: \"4e1fc10d-f0a0-4142-af35-8e86a9e1e037\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:06:20.646127 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.646086    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:06:20.915205 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:20.915178    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"]
Sep 09 21:06:20.916628 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:06:20.916600    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1fc10d_f0a0_4142_af35_8e86a9e1e037.slice/crio-2535f761f2d9cdb36490afc4449bd179800f232ec0842c70889c2916415db39b WatchSource:0}: Error finding container 2535f761f2d9cdb36490afc4449bd179800f232ec0842c70889c2916415db39b: Status 404 returned error can't find the container with id 2535f761f2d9cdb36490afc4449bd179800f232ec0842c70889c2916415db39b
Sep 09 21:06:21.490820 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:21.490785    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv" event={"ID":"4e1fc10d-f0a0-4142-af35-8e86a9e1e037","Type":"ContainerStarted","Data":"f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3"}
Sep 09 21:06:21.490993 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:21.490840    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv" event={"ID":"4e1fc10d-f0a0-4142-af35-8e86a9e1e037","Type":"ContainerStarted","Data":"2535f761f2d9cdb36490afc4449bd179800f232ec0842c70889c2916415db39b"}
Sep 09 21:06:21.490993 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:21.490873    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:06:21.507955 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:21.507905    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv" podStartSLOduration=1.507888978 podStartE2EDuration="1.507888978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:21.505690497 +0000 UTC m=+2041.656362841" watchObservedRunningTime="2026-09-09 21:06:21.507888978 +0000 UTC m=+2041.658561322"
Sep 09 21:06:32.498415 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:32.498386    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:06:36.600593 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:36.600561    2550 generic.go:356] "Generic (PLEG): container finished" podID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerID="052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779" exitCode=0
Sep 09 21:06:36.601012 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:36.600635    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" event={"ID":"3530dd8c-0f36-4309-bdda-5b481c304d02","Type":"ContainerDied","Data":"052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779"}
Sep 09 21:06:36.601060 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:36.601033    2550 scope.go:117] "RemoveContainer" containerID="052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779"
Sep 09 21:06:36.602153 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:36.602127    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:36.602239 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:36.602220    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:36.602278 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:36.602258    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:37.607722 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:37.607683    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" event={"ID":"3530dd8c-0f36-4309-bdda-5b481c304d02","Type":"ContainerStarted","Data":"3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef"}
Sep 09 21:06:38.382958 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:38.382924    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:38.383215 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:38.383192    2550 patch_prober.go:28] interesting pod/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 09 21:06:38.383273 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:38.383251    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 09 21:06:39.383040 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:39.383007    2550 patch_prober.go:28] interesting pod/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 09 21:06:39.383450 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:39.383063    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 09 21:06:40.382383 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:40.382349    2550 patch_prober.go:28] interesting pod/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 09 21:06:40.382554 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:40.382405    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 09 21:06:41.382851 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:41.382820    2550 patch_prober.go:28] interesting pod/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 09 21:06:41.383243 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:41.382868    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 09 21:06:42.382418 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:42.382382    2550 patch_prober.go:28] interesting pod/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 09 21:06:42.382597 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:42.382434    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 09 21:06:43.313555 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:43.313480    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:43.391803 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:43.391771    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:43.392289 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:43.392275    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:43.392963 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:43.392943    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:06:48.181780 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:48.181748    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8n4mh_85871292-25fc-4b5d-ad75-c0df6e970f08/payload-processing/0.log"
Sep 09 21:06:50.008374 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:50.008338    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:51.489888 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.489846    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"]
Sep 09 21:06:51.493132 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.493109    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.495056 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.495037    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-35e9ca\"/\"kube-root-ca.crt\""
Sep 09 21:06:51.495213 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.495037    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-35e9ca\"/\"openshift-service-ca.crt\""
Sep 09 21:06:51.495213 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.495186    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-35e9ca\"/\"ipp-route-35e9ca-kserve-self-signed-certs\""
Sep 09 21:06:51.495349 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.495216    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-35e9ca\"/\"default-dockercfg-wfjbc\""
Sep 09 21:06:51.502710 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.502431    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"]
Sep 09 21:06:51.523506 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.523479    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:51.559017 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.558989    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-home\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.559213 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.559046    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-model-cache\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.559213 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.559104    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tmp-dir\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.559332 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.559213    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-dshm\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.559332 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.559238    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5gph\" (UniqueName: \"kubernetes.io/projected/3577e639-9ac9-4c14-83db-1b5dac6f67d0-kube-api-access-f5gph\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.559404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.559326    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tls-certs\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.659782 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.659754    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tls-certs\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.659782 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.659784    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-home\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.660039 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.659821    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-model-cache\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.660039 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.659848    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tmp-dir\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.660039 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.659901    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-dshm\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.660039 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.660015    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5gph\" (UniqueName: \"kubernetes.io/projected/3577e639-9ac9-4c14-83db-1b5dac6f67d0-kube-api-access-f5gph\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.660285 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.660245    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-home\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.660285 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.660257    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tmp-dir\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.660397 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.660335    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-model-cache\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.662145 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.662121    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-dshm\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.662458 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.662441    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tls-certs\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.670749 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.670727    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5gph\" (UniqueName: \"kubernetes.io/projected/3577e639-9ac9-4c14-83db-1b5dac6f67d0-kube-api-access-f5gph\") pod \"ipp-route-35e9ca-kserve-7d847b8499-ldh5c\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") " pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:51.805988 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:51.805905    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:06:52.085234 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:52.085210    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"]
Sep 09 21:06:52.086568 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:06:52.086540    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3577e639_9ac9_4c14_83db_1b5dac6f67d0.slice/crio-b6d31a7a8ef379564d6d88d6012d5d3626415f13e93d8d0d5f8c956c0cdccde8 WatchSource:0}: Error finding container b6d31a7a8ef379564d6d88d6012d5d3626415f13e93d8d0d5f8c956c0cdccde8: Status 404 returned error can't find the container with id b6d31a7a8ef379564d6d88d6012d5d3626415f13e93d8d0d5f8c956c0cdccde8
Sep 09 21:06:52.690638 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:52.690604    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c" event={"ID":"3577e639-9ac9-4c14-83db-1b5dac6f67d0","Type":"ContainerStarted","Data":"57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55"}
Sep 09 21:06:52.691010 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:52.690648    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c" event={"ID":"3577e639-9ac9-4c14-83db-1b5dac6f67d0","Type":"ContainerStarted","Data":"b6d31a7a8ef379564d6d88d6012d5d3626415f13e93d8d0d5f8c956c0cdccde8"}
Sep 09 21:06:52.708558 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:06:52.708512    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c" podStartSLOduration=1.708495773 podStartE2EDuration="1.708495773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:52.706280367 +0000 UTC m=+2072.856952717" watchObservedRunningTime="2026-09-09 21:06:52.708495773 +0000 UTC m=+2072.859168150"
Sep 09 21:07:01.806820 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:01.806785    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:07:01.807210 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:01.806833    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:07:01.819206 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:01.819188    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:07:02.761688 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:02.761656    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:07:14.653789 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:14.653760    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8_048a5687-42a4-4b44-beee-db5bd2192a1f/payload-processing/0.log"
Sep 09 21:07:14.779143 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:14.779113    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8n4mh_85871292-25fc-4b5d-ad75-c0df6e970f08/payload-processing/0.log"
Sep 09 21:07:15.176558 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:15.176526    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:15.475522 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:15.475491    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:15.689454 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:15.689425    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:18.821771 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:18.821734    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:18.952841 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:18.952804    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:20.025715 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.025683    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:20.126082 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.126041    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:20.282648 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.282567    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:20.470385 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.470356    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 21:07:20.480733 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.480704    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 21:07:20.603213 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.603113    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:20.706896 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.706862    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:20.805115 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.805078    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"]
Sep 09 21:07:20.807844 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.807823    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:20.810747 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.810722    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-334ee7\""
Sep 09 21:07:20.810886 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.810722    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-334ee7-dockercfg-smm7v\""
Sep 09 21:07:20.822582 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.822556    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"]
Sep 09 21:07:20.921433 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.921351    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsm85\" (UniqueName: \"kubernetes.io/projected/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-kube-api-access-xsm85\") pod \"payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl\" (UID: \"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:20.921433 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:20.921401    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl\" (UID: \"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:21.021969 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:21.021930    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl\" (UID: \"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:21.022207 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:21.022114    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsm85\" (UniqueName: \"kubernetes.io/projected/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-kube-api-access-xsm85\") pod \"payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl\" (UID: \"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:21.022748 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:21.022720    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl\" (UID: \"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:21.033826 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:21.033794    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsm85\" (UniqueName: \"kubernetes.io/projected/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-kube-api-access-xsm85\") pod \"payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl\" (UID: \"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:21.118411 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:21.118378    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:21.393326 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:21.393295    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"]
Sep 09 21:07:21.394430 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:07:21.394403    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6a0e5d_1282_46c3_9b48_3faa54c5cbb2.slice/crio-4de98bb170b4eefefd1e8a8a1ca6199ed233bd75fc9b36358b41f32dd50008d3 WatchSource:0}: Error finding container 4de98bb170b4eefefd1e8a8a1ca6199ed233bd75fc9b36358b41f32dd50008d3: Status 404 returned error can't find the container with id 4de98bb170b4eefefd1e8a8a1ca6199ed233bd75fc9b36358b41f32dd50008d3
Sep 09 21:07:21.860221 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:21.860184    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl" event={"ID":"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2","Type":"ContainerStarted","Data":"bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e"}
Sep 09 21:07:21.860406 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:21.860237    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl" event={"ID":"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2","Type":"ContainerStarted","Data":"4de98bb170b4eefefd1e8a8a1ca6199ed233bd75fc9b36358b41f32dd50008d3"}
Sep 09 21:07:21.860406 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:21.860379    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:21.877429 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:21.877376    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl" podStartSLOduration=1.877360827 podStartE2EDuration="1.877360827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:07:21.87583525 +0000 UTC m=+2102.026507605" watchObservedRunningTime="2026-09-09 21:07:21.877360827 +0000 UTC m=+2102.028033170"
Sep 09 21:07:32.868770 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:32.868737    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:43.800141 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:43.800109    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:50.636746 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:50.636703    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:50.670293 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:50.670259    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"]
Sep 09 21:07:50.670539 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:50.670514    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl" podUID="aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2" containerName="payload-pre-processing" containerID="cri-o://bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e" gracePeriod=30
Sep 09 21:07:50.738485 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:50.738448    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:50.837386 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:50.837350    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:50.936859 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:50.936829    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:50.941993 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:50.941970    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:51.033537 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.033500    2550 generic.go:356] "Generic (PLEG): container finished" podID="aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2" containerID="bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e" exitCode=0
Sep 09 21:07:51.033739 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.033576    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"
Sep 09 21:07:51.033739 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.033613    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl" event={"ID":"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2","Type":"ContainerDied","Data":"bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e"}
Sep 09 21:07:51.033739 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.033672    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl" event={"ID":"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2","Type":"ContainerDied","Data":"4de98bb170b4eefefd1e8a8a1ca6199ed233bd75fc9b36358b41f32dd50008d3"}
Sep 09 21:07:51.033739 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.033703    2550 scope.go:117] "RemoveContainer" containerID="bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e"
Sep 09 21:07:51.043600 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.043582    2550 scope.go:117] "RemoveContainer" containerID="bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e"
Sep 09 21:07:51.043856 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:07:51.043836    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e\": container with ID starting with bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e not found: ID does not exist" containerID="bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e"
Sep 09 21:07:51.043920 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.043868    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e"} err="failed to get container status \"bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e\": rpc error: code = NotFound desc = could not find container \"bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e\": container with ID starting with bf0066c36f09d054dece5544179313ffcb494f7e54fdd3de40558e35c304023e not found: ID does not exist"
Sep 09 21:07:51.092283 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.092256    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-plugins-config-volume\") pod \"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2\" (UID: \"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2\") "
Sep 09 21:07:51.092395 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.092311    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsm85\" (UniqueName: \"kubernetes.io/projected/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-kube-api-access-xsm85\") pod \"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2\" (UID: \"aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2\") "
Sep 09 21:07:51.092640 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.092620    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2" (UID: "aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:51.094342 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.094310    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-kube-api-access-xsm85" (OuterVolumeSpecName: "kube-api-access-xsm85") pod "aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2" (UID: "aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2"). InnerVolumeSpecName "kube-api-access-xsm85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:51.193368 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.193290    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:07:51.193368 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.193317    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsm85\" (UniqueName: \"kubernetes.io/projected/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2-kube-api-access-xsm85\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:07:51.195678 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.195640    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:51.354787 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.354761    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"]
Sep 09 21:07:51.360915 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:51.360888    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-334ee7-6fc9dc6c94-cw9gl"]
Sep 09 21:07:52.279769 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:52.279733    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2" path="/var/lib/kubelet/pods/aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2/volumes"
Sep 09 21:07:57.428571 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:57.428497    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:57.497141 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:57.497106    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:57.723808 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:07:57.723780    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:01.333904 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:01.333867    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:16.711422 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:16.711326    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:16.737715 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:16.737683    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"]
Sep 09 21:08:16.738202 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:16.738147    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8" podUID="048a5687-42a4-4b44-beee-db5bd2192a1f" containerName="payload-processing" containerID="cri-o://f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa" gracePeriod=30
Sep 09 21:08:16.814421 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:16.814385    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:16.913486 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:16.913456    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:16.982883 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:16.982862    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:08:17.007726 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.007691    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:17.022679 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.022654    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6bzd\" (UniqueName: \"kubernetes.io/projected/048a5687-42a4-4b44-beee-db5bd2192a1f-kube-api-access-p6bzd\") pod \"048a5687-42a4-4b44-beee-db5bd2192a1f\" (UID: \"048a5687-42a4-4b44-beee-db5bd2192a1f\") "
Sep 09 21:08:17.022821 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.022742    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/048a5687-42a4-4b44-beee-db5bd2192a1f-plugins-config-volume\") pod \"048a5687-42a4-4b44-beee-db5bd2192a1f\" (UID: \"048a5687-42a4-4b44-beee-db5bd2192a1f\") "
Sep 09 21:08:17.023202 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.023155    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/048a5687-42a4-4b44-beee-db5bd2192a1f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "048a5687-42a4-4b44-beee-db5bd2192a1f" (UID: "048a5687-42a4-4b44-beee-db5bd2192a1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:17.024770 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.024750    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/048a5687-42a4-4b44-beee-db5bd2192a1f-kube-api-access-p6bzd" (OuterVolumeSpecName: "kube-api-access-p6bzd") pod "048a5687-42a4-4b44-beee-db5bd2192a1f" (UID: "048a5687-42a4-4b44-beee-db5bd2192a1f"). InnerVolumeSpecName "kube-api-access-p6bzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:17.123473 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.123439    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6bzd\" (UniqueName: \"kubernetes.io/projected/048a5687-42a4-4b44-beee-db5bd2192a1f-kube-api-access-p6bzd\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:08:17.123473 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.123467    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/048a5687-42a4-4b44-beee-db5bd2192a1f-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:08:17.185799 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.185767    2550 generic.go:356] "Generic (PLEG): container finished" podID="048a5687-42a4-4b44-beee-db5bd2192a1f" containerID="f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa" exitCode=0
Sep 09 21:08:17.185954 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.185813    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"
Sep 09 21:08:17.185954 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.185848    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8" event={"ID":"048a5687-42a4-4b44-beee-db5bd2192a1f","Type":"ContainerDied","Data":"f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa"}
Sep 09 21:08:17.185954 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.185898    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8" event={"ID":"048a5687-42a4-4b44-beee-db5bd2192a1f","Type":"ContainerDied","Data":"a23d58a36cca14e8adc8cb2783baa16abd58c39ec974bf0c229d869e7f4067b1"}
Sep 09 21:08:17.185954 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.185922    2550 scope.go:117] "RemoveContainer" containerID="f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa"
Sep 09 21:08:17.197429 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.197412    2550 scope.go:117] "RemoveContainer" containerID="f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa"
Sep 09 21:08:17.197665 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:08:17.197642    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa\": container with ID starting with f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa not found: ID does not exist" containerID="f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa"
Sep 09 21:08:17.197721 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.197674    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa"} err="failed to get container status \"f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa\": rpc error: code = NotFound desc = could not find container \"f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa\": container with ID starting with f417ae5a2d3d1adab729fb9449156f1f5e57053977fc7fc7f84652c94c235faa not found: ID does not exist"
Sep 09 21:08:17.212550 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.212525    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"]
Sep 09 21:08:17.216059 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.216011    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-35e9ca-865685f9b9-9wpt8"]
Sep 09 21:08:17.315229 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:17.315196    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:18.280642 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:18.280610    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="048a5687-42a4-4b44-beee-db5bd2192a1f" path="/var/lib/kubelet/pods/048a5687-42a4-4b44-beee-db5bd2192a1f/volumes"
Sep 09 21:08:22.288456 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:22.288418    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"]
Sep 09 21:08:22.340686 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:22.340645    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:37.308160 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.308120    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c" podUID="3577e639-9ac9-4c14-83db-1b5dac6f67d0" containerName="main" containerID="cri-o://57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55" gracePeriod=45
Sep 09 21:08:37.561045 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.560983    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:08:37.610884 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.610850    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-model-cache\") pod \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") "
Sep 09 21:08:37.611056 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.610903    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5gph\" (UniqueName: \"kubernetes.io/projected/3577e639-9ac9-4c14-83db-1b5dac6f67d0-kube-api-access-f5gph\") pod \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") "
Sep 09 21:08:37.611056 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.610931    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tls-certs\") pod \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") "
Sep 09 21:08:37.611056 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.610955    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tmp-dir\") pod \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") "
Sep 09 21:08:37.611056 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.610994    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-home\") pod \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") "
Sep 09 21:08:37.611056 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.611042    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-dshm\") pod \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\" (UID: \"3577e639-9ac9-4c14-83db-1b5dac6f67d0\") "
Sep 09 21:08:37.611345 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.611149    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-model-cache" (OuterVolumeSpecName: "model-cache") pod "3577e639-9ac9-4c14-83db-1b5dac6f67d0" (UID: "3577e639-9ac9-4c14-83db-1b5dac6f67d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:37.611402 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.611387    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:08:37.611509 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.611478    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3577e639-9ac9-4c14-83db-1b5dac6f67d0" (UID: "3577e639-9ac9-4c14-83db-1b5dac6f67d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:37.611599 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.611548    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-home" (OuterVolumeSpecName: "home") pod "3577e639-9ac9-4c14-83db-1b5dac6f67d0" (UID: "3577e639-9ac9-4c14-83db-1b5dac6f67d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:37.613294 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.613265    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-dshm" (OuterVolumeSpecName: "dshm") pod "3577e639-9ac9-4c14-83db-1b5dac6f67d0" (UID: "3577e639-9ac9-4c14-83db-1b5dac6f67d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:37.613294 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.613265    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3577e639-9ac9-4c14-83db-1b5dac6f67d0" (UID: "3577e639-9ac9-4c14-83db-1b5dac6f67d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:08:37.613484 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.613465    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3577e639-9ac9-4c14-83db-1b5dac6f67d0-kube-api-access-f5gph" (OuterVolumeSpecName: "kube-api-access-f5gph") pod "3577e639-9ac9-4c14-83db-1b5dac6f67d0" (UID: "3577e639-9ac9-4c14-83db-1b5dac6f67d0"). InnerVolumeSpecName "kube-api-access-f5gph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:37.712094 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.712057    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5gph\" (UniqueName: \"kubernetes.io/projected/3577e639-9ac9-4c14-83db-1b5dac6f67d0-kube-api-access-f5gph\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:08:37.712094 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.712087    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:08:37.712319 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.712117    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:08:37.712319 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.712125    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:08:37.712319 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:37.712133    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3577e639-9ac9-4c14-83db-1b5dac6f67d0-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:08:38.307462 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:38.307430    2550 generic.go:356] "Generic (PLEG): container finished" podID="3577e639-9ac9-4c14-83db-1b5dac6f67d0" containerID="57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55" exitCode=0
Sep 09 21:08:38.307643 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:38.307480    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"
Sep 09 21:08:38.307643 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:38.307504    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c" event={"ID":"3577e639-9ac9-4c14-83db-1b5dac6f67d0","Type":"ContainerDied","Data":"57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55"}
Sep 09 21:08:38.307643 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:38.307539    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c" event={"ID":"3577e639-9ac9-4c14-83db-1b5dac6f67d0","Type":"ContainerDied","Data":"b6d31a7a8ef379564d6d88d6012d5d3626415f13e93d8d0d5f8c956c0cdccde8"}
Sep 09 21:08:38.307643 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:38.307559    2550 scope.go:117] "RemoveContainer" containerID="57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55"
Sep 09 21:08:38.317078 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:38.317061    2550 scope.go:117] "RemoveContainer" containerID="57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55"
Sep 09 21:08:38.317412 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:08:38.317389    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55\": container with ID starting with 57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55 not found: ID does not exist" containerID="57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55"
Sep 09 21:08:38.317458 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:38.317424    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55"} err="failed to get container status \"57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55\": rpc error: code = NotFound desc = could not find container \"57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55\": container with ID starting with 57d4ccc184c1689e4eb24d3591f9bdb2fd346268eafab8561d2ca52ef66ebf55 not found: ID does not exist"
Sep 09 21:08:38.324658 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:38.324633    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"]
Sep 09 21:08:38.328221 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:38.328200    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-35e9ca/ipp-route-35e9ca-kserve-7d847b8499-ldh5c"]
Sep 09 21:08:40.279732 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:08:40.279701    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3577e639-9ac9-4c14-83db-1b5dac6f67d0" path="/var/lib/kubelet/pods/3577e639-9ac9-4c14-83db-1b5dac6f67d0/volumes"
Sep 09 21:09:06.595638 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:06.595605    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:07.445567 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:07.445529    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:13.057926 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:13.057895    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:17.576672 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:17.576635    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:27.401850 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:27.401814    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:27.476727 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:27.476696    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:27.643624 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:27.643589    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"]
Sep 09 21:09:27.643899 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:27.643862    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy" containerID="cri-o://3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493" gracePeriod=30
Sep 09 21:09:27.672084 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:27.672006    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:28.249566 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:28.249533    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:32.898268 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.898247    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:09:32.977078 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977051    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-certs\") pod \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") "
Sep 09 21:09:32.977277 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977121    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m79lx\" (UniqueName: \"kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-kube-api-access-m79lx\") pod \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") "
Sep 09 21:09:32.977277 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977200    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-data\") pod \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") "
Sep 09 21:09:32.977277 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977240    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-token\") pod \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") "
Sep 09 21:09:32.977277 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977283    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istiod-ca-cert\") pod \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") "
Sep 09 21:09:32.977494 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977311    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-credential-socket\") pod \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") "
Sep 09 21:09:32.977494 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977348    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-envoy\") pod \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") "
Sep 09 21:09:32.977494 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977434    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-podinfo\") pod \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") "
Sep 09 21:09:32.977494 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977357    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" (UID: "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:32.977494 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977469    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-socket\") pod \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\" (UID: \"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e\") "
Sep 09 21:09:32.977735 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977516    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-data" (OuterVolumeSpecName: "istio-data") pod "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" (UID: "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:32.977735 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977613    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" (UID: "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:32.977735 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977636    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" (UID: "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:32.977880 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977833    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" (UID: "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:32.977940 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977923    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:32.977977 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977947    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:32.977977 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977961    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:32.978045 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977976    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:32.978045 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.977990    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:32.979483 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.979439    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" (UID: "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:32.979594 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.979520    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" (UID: "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:09:32.979918 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.979900    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-kube-api-access-m79lx" (OuterVolumeSpecName: "kube-api-access-m79lx") pod "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" (UID: "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e"). InnerVolumeSpecName "kube-api-access-m79lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:32.979977 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:32.979913    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-token" (OuterVolumeSpecName: "istio-token") pod "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" (UID: "3eaae6c9-9519-4924-8c3b-a60e5bcaa79e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:33.078425 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.078389    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:33.078425 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.078421    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:33.078425 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.078433    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:33.078662 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.078446    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m79lx\" (UniqueName: \"kubernetes.io/projected/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e-kube-api-access-m79lx\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:33.617158 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.617123    2550 generic.go:356] "Generic (PLEG): container finished" podID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerID="3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493" exitCode=0
Sep 09 21:09:33.617378 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.617203    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"
Sep 09 21:09:33.617378 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.617209    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" event={"ID":"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e","Type":"ContainerDied","Data":"3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493"}
Sep 09 21:09:33.617378 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.617280    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5" event={"ID":"3eaae6c9-9519-4924-8c3b-a60e5bcaa79e","Type":"ContainerDied","Data":"f1b924bf3d23a9a5b614ffee97928fa73a52bf22c1ad4debb7362af91a35a0bc"}
Sep 09 21:09:33.617378 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.617312    2550 scope.go:117] "RemoveContainer" containerID="3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493"
Sep 09 21:09:33.626728 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.626709    2550 scope.go:117] "RemoveContainer" containerID="67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e"
Sep 09 21:09:33.635006 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.634987    2550 scope.go:117] "RemoveContainer" containerID="3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493"
Sep 09 21:09:33.635271 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:09:33.635251    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493\": container with ID starting with 3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493 not found: ID does not exist" containerID="3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493"
Sep 09 21:09:33.635332 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.635280    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493"} err="failed to get container status \"3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493\": rpc error: code = NotFound desc = could not find container \"3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493\": container with ID starting with 3d8c4d1c74ef83706678e23e2c74e0286dca224f1b6d666337ad8b9611963493 not found: ID does not exist"
Sep 09 21:09:33.635332 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.635296    2550 scope.go:117] "RemoveContainer" containerID="67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e"
Sep 09 21:09:33.635524 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:09:33.635510    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e\": container with ID starting with 67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e not found: ID does not exist" containerID="67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e"
Sep 09 21:09:33.635564 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.635527    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e"} err="failed to get container status \"67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e\": rpc error: code = NotFound desc = could not find container \"67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e\": container with ID starting with 67699802f2ca70d13af75873f0ad77c06667f52d28a9e5c19877dec97b314e4e not found: ID does not exist"
Sep 09 21:09:33.639297 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.639266    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"]
Sep 09 21:09:33.645848 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:33.645826    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-35e9ca-openshift-default-7b47c9798b-hlsr5"]
Sep 09 21:09:34.280010 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:34.279979    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" path="/var/lib/kubelet/pods/3eaae6c9-9519-4924-8c3b-a60e5bcaa79e/volumes"
Sep 09 21:09:38.757788 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:38.757748    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:38.790802 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:38.790773    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"]
Sep 09 21:09:38.791007 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:38.790979    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv" podUID="4e1fc10d-f0a0-4142-af35-8e86a9e1e037" containerName="payload-pre-processing" containerID="cri-o://f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3" gracePeriod=30
Sep 09 21:09:38.855929 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:38.855890    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:38.963729 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:38.963696    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:39.051415 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.051390    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:09:39.060080 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.060053    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:39.134578 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.134545    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f88pw\" (UniqueName: \"kubernetes.io/projected/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-kube-api-access-f88pw\") pod \"4e1fc10d-f0a0-4142-af35-8e86a9e1e037\" (UID: \"4e1fc10d-f0a0-4142-af35-8e86a9e1e037\") "
Sep 09 21:09:39.134787 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.134673    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-plugins-config-volume\") pod \"4e1fc10d-f0a0-4142-af35-8e86a9e1e037\" (UID: \"4e1fc10d-f0a0-4142-af35-8e86a9e1e037\") "
Sep 09 21:09:39.135418 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.135375    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4e1fc10d-f0a0-4142-af35-8e86a9e1e037" (UID: "4e1fc10d-f0a0-4142-af35-8e86a9e1e037"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:39.139114 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.139043    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-kube-api-access-f88pw" (OuterVolumeSpecName: "kube-api-access-f88pw") pod "4e1fc10d-f0a0-4142-af35-8e86a9e1e037" (UID: "4e1fc10d-f0a0-4142-af35-8e86a9e1e037"). InnerVolumeSpecName "kube-api-access-f88pw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:39.235888 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.235861    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f88pw\" (UniqueName: \"kubernetes.io/projected/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-kube-api-access-f88pw\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:39.235888 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.235885    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e1fc10d-f0a0-4142-af35-8e86a9e1e037-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:39.377218 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.377110    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:39.653800 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.653715    2550 generic.go:356] "Generic (PLEG): container finished" podID="4e1fc10d-f0a0-4142-af35-8e86a9e1e037" containerID="f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3" exitCode=0
Sep 09 21:09:39.653800 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.653754    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"
Sep 09 21:09:39.653800 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.653793    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv" event={"ID":"4e1fc10d-f0a0-4142-af35-8e86a9e1e037","Type":"ContainerDied","Data":"f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3"}
Sep 09 21:09:39.654056 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.653843    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv" event={"ID":"4e1fc10d-f0a0-4142-af35-8e86a9e1e037","Type":"ContainerDied","Data":"2535f761f2d9cdb36490afc4449bd179800f232ec0842c70889c2916415db39b"}
Sep 09 21:09:39.654056 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.653866    2550 scope.go:117] "RemoveContainer" containerID="f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3"
Sep 09 21:09:39.663341 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.663272    2550 scope.go:117] "RemoveContainer" containerID="f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3"
Sep 09 21:09:39.663537 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:09:39.663517    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3\": container with ID starting with f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3 not found: ID does not exist" containerID="f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3"
Sep 09 21:09:39.663604 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.663548    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3"} err="failed to get container status \"f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3\": rpc error: code = NotFound desc = could not find container \"f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3\": container with ID starting with f8e4ce65866f39039ced97c9011c4535ae7bc0ca00cbc39a06826c6b79ee87c3 not found: ID does not exist"
Sep 09 21:09:39.676066 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.676043    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"]
Sep 09 21:09:39.680885 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:39.680862    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2ff038-6c87474886-pk5lv"]
Sep 09 21:09:40.279743 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:40.279714    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e1fc10d-f0a0-4142-af35-8e86a9e1e037" path="/var/lib/kubelet/pods/4e1fc10d-f0a0-4142-af35-8e86a9e1e037/volumes"
Sep 09 21:09:45.312453 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:45.312365    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:45.633239 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:45.633135    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"]
Sep 09 21:09:45.633414 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:45.633362    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy" containerID="cri-o://3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef" gracePeriod=30
Sep 09 21:09:46.714857 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:46.714819    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:50.880925 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:50.880901    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:09:51.041284 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041253    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-socket\") pod \"3530dd8c-0f36-4309-bdda-5b481c304d02\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") "
Sep 09 21:09:51.041470 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041300    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bfj8\" (UniqueName: \"kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-kube-api-access-6bfj8\") pod \"3530dd8c-0f36-4309-bdda-5b481c304d02\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") "
Sep 09 21:09:51.041470 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041338    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-envoy\") pod \"3530dd8c-0f36-4309-bdda-5b481c304d02\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") "
Sep 09 21:09:51.041470 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041364    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-credential-socket\") pod \"3530dd8c-0f36-4309-bdda-5b481c304d02\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") "
Sep 09 21:09:51.041470 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041380    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-podinfo\") pod \"3530dd8c-0f36-4309-bdda-5b481c304d02\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") "
Sep 09 21:09:51.041470 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041409    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-certs\") pod \"3530dd8c-0f36-4309-bdda-5b481c304d02\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") "
Sep 09 21:09:51.041740 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041559    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3530dd8c-0f36-4309-bdda-5b481c304d02" (UID: "3530dd8c-0f36-4309-bdda-5b481c304d02"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:51.041740 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041586    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-data\") pod \"3530dd8c-0f36-4309-bdda-5b481c304d02\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") "
Sep 09 21:09:51.041740 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041624    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-token\") pod \"3530dd8c-0f36-4309-bdda-5b481c304d02\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") "
Sep 09 21:09:51.041740 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041698    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3530dd8c-0f36-4309-bdda-5b481c304d02-istiod-ca-cert\") pod \"3530dd8c-0f36-4309-bdda-5b481c304d02\" (UID: \"3530dd8c-0f36-4309-bdda-5b481c304d02\") "
Sep 09 21:09:51.042005 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.041982    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:51.042185 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.042146    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3530dd8c-0f36-4309-bdda-5b481c304d02" (UID: "3530dd8c-0f36-4309-bdda-5b481c304d02"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:51.042271 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.042156    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3530dd8c-0f36-4309-bdda-5b481c304d02" (UID: "3530dd8c-0f36-4309-bdda-5b481c304d02"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:51.042338 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.042268    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-data" (OuterVolumeSpecName: "istio-data") pod "3530dd8c-0f36-4309-bdda-5b481c304d02" (UID: "3530dd8c-0f36-4309-bdda-5b481c304d02"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:51.042404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.042380    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3530dd8c-0f36-4309-bdda-5b481c304d02-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3530dd8c-0f36-4309-bdda-5b481c304d02" (UID: "3530dd8c-0f36-4309-bdda-5b481c304d02"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:51.044039 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.044005    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3530dd8c-0f36-4309-bdda-5b481c304d02" (UID: "3530dd8c-0f36-4309-bdda-5b481c304d02"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:09:51.044138 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.044049    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-kube-api-access-6bfj8" (OuterVolumeSpecName: "kube-api-access-6bfj8") pod "3530dd8c-0f36-4309-bdda-5b481c304d02" (UID: "3530dd8c-0f36-4309-bdda-5b481c304d02"). InnerVolumeSpecName "kube-api-access-6bfj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:51.044138 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.044076    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3530dd8c-0f36-4309-bdda-5b481c304d02" (UID: "3530dd8c-0f36-4309-bdda-5b481c304d02"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:51.044240 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.044135    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-token" (OuterVolumeSpecName: "istio-token") pod "3530dd8c-0f36-4309-bdda-5b481c304d02" (UID: "3530dd8c-0f36-4309-bdda-5b481c304d02"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:51.142721 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.142681    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3530dd8c-0f36-4309-bdda-5b481c304d02-istiod-ca-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:51.142898 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.142728    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bfj8\" (UniqueName: \"kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-kube-api-access-6bfj8\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:51.142898 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.142742    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-envoy\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:51.142898 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.142754    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-credential-socket\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:51.142898 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.142767    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-podinfo\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:51.142898 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.142778    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-workload-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:51.142898 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.142790    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-data\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:51.142898 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.142799    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3530dd8c-0f36-4309-bdda-5b481c304d02-istio-token\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:51.326991 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.326889    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:51.730804 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.730773    2550 generic.go:356] "Generic (PLEG): container finished" podID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerID="3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef" exitCode=0
Sep 09 21:09:51.731003 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.730823    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"
Sep 09 21:09:51.731003 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.730834    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" event={"ID":"3530dd8c-0f36-4309-bdda-5b481c304d02","Type":"ContainerDied","Data":"3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef"}
Sep 09 21:09:51.731003 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.730890    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n" event={"ID":"3530dd8c-0f36-4309-bdda-5b481c304d02","Type":"ContainerDied","Data":"5f6af0166bec00ec08d39dc7747a888c6580aef36d4bc626de372a267de3c170"}
Sep 09 21:09:51.731003 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.730919    2550 scope.go:117] "RemoveContainer" containerID="3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef"
Sep 09 21:09:51.740835 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.740817    2550 scope.go:117] "RemoveContainer" containerID="052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779"
Sep 09 21:09:51.748869 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.748853    2550 scope.go:117] "RemoveContainer" containerID="3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef"
Sep 09 21:09:51.749157 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:09:51.749139    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef\": container with ID starting with 3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef not found: ID does not exist" containerID="3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef"
Sep 09 21:09:51.749217 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.749181    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef"} err="failed to get container status \"3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef\": rpc error: code = NotFound desc = could not find container \"3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef\": container with ID starting with 3a6db0e5ea6ac4f3413a23f0b1b4c70b630a82adf3e16e25f7dce5f13d9261ef not found: ID does not exist"
Sep 09 21:09:51.749217 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.749205    2550 scope.go:117] "RemoveContainer" containerID="052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779"
Sep 09 21:09:51.749483 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:09:51.749463    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779\": container with ID starting with 052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779 not found: ID does not exist" containerID="052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779"
Sep 09 21:09:51.749522 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.749489    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779"} err="failed to get container status \"052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779\": rpc error: code = NotFound desc = could not find container \"052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779\": container with ID starting with 052d287b045a9b2925fa2ab816f6d12cda4c43284a57e153078b5417bcef0779 not found: ID does not exist"
Sep 09 21:09:51.753030 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.753007    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"]
Sep 09 21:09:51.756961 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:51.756943    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-2ff038-openshift-default-5bf696b9fd-j6p6n"]
Sep 09 21:09:52.279802 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:52.279766    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" path="/var/lib/kubelet/pods/3530dd8c-0f36-4309-bdda-5b481c304d02/volumes"
Sep 09 21:09:56.207175 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:56.207141    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:56.830595 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:56.830560    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"]
Sep 09 21:09:56.830866 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:56.830827    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9" podUID="ef417d3c-c927-4700-9789-05cb639a76d1" containerName="payload-pre-processing" containerID="cri-o://2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e" gracePeriod=30
Sep 09 21:09:56.840880 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:56.840852    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:56.937151 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:56.937114    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:57.037505 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.037476    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:57.091822 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.091766    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 21:09:57.105943 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.105916    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:57.187836 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.187813    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5jzc\" (UniqueName: \"kubernetes.io/projected/ef417d3c-c927-4700-9789-05cb639a76d1-kube-api-access-b5jzc\") pod \"ef417d3c-c927-4700-9789-05cb639a76d1\" (UID: \"ef417d3c-c927-4700-9789-05cb639a76d1\") "
Sep 09 21:09:57.187995 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.187876    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef417d3c-c927-4700-9789-05cb639a76d1-plugins-config-volume\") pod \"ef417d3c-c927-4700-9789-05cb639a76d1\" (UID: \"ef417d3c-c927-4700-9789-05cb639a76d1\") "
Sep 09 21:09:57.188287 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.188264    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef417d3c-c927-4700-9789-05cb639a76d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef417d3c-c927-4700-9789-05cb639a76d1" (UID: "ef417d3c-c927-4700-9789-05cb639a76d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:57.189871 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.189843    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef417d3c-c927-4700-9789-05cb639a76d1-kube-api-access-b5jzc" (OuterVolumeSpecName: "kube-api-access-b5jzc") pod "ef417d3c-c927-4700-9789-05cb639a76d1" (UID: "ef417d3c-c927-4700-9789-05cb639a76d1"). InnerVolumeSpecName "kube-api-access-b5jzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:57.288893 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.288866    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef417d3c-c927-4700-9789-05cb639a76d1-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:57.288893 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.288890    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5jzc\" (UniqueName: \"kubernetes.io/projected/ef417d3c-c927-4700-9789-05cb639a76d1-kube-api-access-b5jzc\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:09:57.444868 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.444780    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:57.764611 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.764574    2550 generic.go:356] "Generic (PLEG): container finished" podID="ef417d3c-c927-4700-9789-05cb639a76d1" containerID="2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e" exitCode=0
Sep 09 21:09:57.764808 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.764619    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"
Sep 09 21:09:57.764808 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.764656    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9" event={"ID":"ef417d3c-c927-4700-9789-05cb639a76d1","Type":"ContainerDied","Data":"2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e"}
Sep 09 21:09:57.764808 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.764705    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9" event={"ID":"ef417d3c-c927-4700-9789-05cb639a76d1","Type":"ContainerDied","Data":"7c5ba1a5324145d6c8e1a285b6eda51caf316b8446dbd8af9b94a8470c0e9847"}
Sep 09 21:09:57.764808 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.764718    2550 scope.go:117] "RemoveContainer" containerID="2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e"
Sep 09 21:09:57.773700 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.773682    2550 scope.go:117] "RemoveContainer" containerID="2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e"
Sep 09 21:09:57.773954 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:09:57.773936    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e\": container with ID starting with 2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e not found: ID does not exist" containerID="2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e"
Sep 09 21:09:57.774020 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.773960    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e"} err="failed to get container status \"2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e\": rpc error: code = NotFound desc = could not find container \"2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e\": container with ID starting with 2adec02b2034878eda520e8746bdff84bbc12baabce7a579bf8d21beb2188e0e not found: ID does not exist"
Sep 09 21:09:57.786978 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.786957    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"]
Sep 09 21:09:57.791743 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:57.791719    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cb82f2-997897b7d-7f6x9"]
Sep 09 21:09:58.279571 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:09:58.279542    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef417d3c-c927-4700-9789-05cb639a76d1" path="/var/lib/kubelet/pods/ef417d3c-c927-4700-9789-05cb639a76d1/volumes"
Sep 09 21:10:03.672447 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:03.672412    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:03.741861 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:03.741825    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:03.957466 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:03.957376    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:04.395304 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:04.395271    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:14.804796 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:14.804761    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws"]
Sep 09 21:10:14.813777 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:14.813736    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a2fb5a-29816460-lxbws"]
Sep 09 21:10:14.815185 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:14.815134    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:14.837072 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:14.837045    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"]
Sep 09 21:10:14.837281 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:14.837260    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk" podUID="c15e5119-cb46-486a-8b6e-d5a2db44b9ef" containerName="payload-processing" containerID="cri-o://94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72" gracePeriod=30
Sep 09 21:10:14.847593 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:14.847569    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"]
Sep 09 21:10:14.848046 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:14.848016    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc" podUID="69effffe-d680-426a-9518-031affe8ad54" containerName="payload-pre-processing" containerID="cri-o://25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c" gracePeriod=30
Sep 09 21:10:14.892578 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:14.892534    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:14.972440 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:14.972412    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:15.076010 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.075873    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:15.127535 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.127516    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 21:10:15.129551 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.129531    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-plugins-config-volume\") pod \"c15e5119-cb46-486a-8b6e-d5a2db44b9ef\" (UID: \"c15e5119-cb46-486a-8b6e-d5a2db44b9ef\") "
Sep 09 21:10:15.129658 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.129559    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-289k6\" (UniqueName: \"kubernetes.io/projected/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-kube-api-access-289k6\") pod \"c15e5119-cb46-486a-8b6e-d5a2db44b9ef\" (UID: \"c15e5119-cb46-486a-8b6e-d5a2db44b9ef\") "
Sep 09 21:10:15.129906 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.129882    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c15e5119-cb46-486a-8b6e-d5a2db44b9ef" (UID: "c15e5119-cb46-486a-8b6e-d5a2db44b9ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:10:15.130399 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.130286    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 21:10:15.131948 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.131924    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-kube-api-access-289k6" (OuterVolumeSpecName: "kube-api-access-289k6") pod "c15e5119-cb46-486a-8b6e-d5a2db44b9ef" (UID: "c15e5119-cb46-486a-8b6e-d5a2db44b9ef"). InnerVolumeSpecName "kube-api-access-289k6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:10:15.153744 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.153715    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:15.230139 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.230113    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69effffe-d680-426a-9518-031affe8ad54-plugins-config-volume\") pod \"69effffe-d680-426a-9518-031affe8ad54\" (UID: \"69effffe-d680-426a-9518-031affe8ad54\") "
Sep 09 21:10:15.230345 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.230213    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxvzh\" (UniqueName: \"kubernetes.io/projected/69effffe-d680-426a-9518-031affe8ad54-kube-api-access-qxvzh\") pod \"69effffe-d680-426a-9518-031affe8ad54\" (UID: \"69effffe-d680-426a-9518-031affe8ad54\") "
Sep 09 21:10:15.230481 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.230464    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:10:15.230566 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.230485    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-289k6\" (UniqueName: \"kubernetes.io/projected/c15e5119-cb46-486a-8b6e-d5a2db44b9ef-kube-api-access-289k6\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:10:15.230566 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.230498    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69effffe-d680-426a-9518-031affe8ad54-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69effffe-d680-426a-9518-031affe8ad54" (UID: "69effffe-d680-426a-9518-031affe8ad54"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:10:15.232548 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.232518    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69effffe-d680-426a-9518-031affe8ad54-kube-api-access-qxvzh" (OuterVolumeSpecName: "kube-api-access-qxvzh") pod "69effffe-d680-426a-9518-031affe8ad54" (UID: "69effffe-d680-426a-9518-031affe8ad54"). InnerVolumeSpecName "kube-api-access-qxvzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:10:15.331639 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.331565    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxvzh\" (UniqueName: \"kubernetes.io/projected/69effffe-d680-426a-9518-031affe8ad54-kube-api-access-qxvzh\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:10:15.331639 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.331597    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69effffe-d680-426a-9518-031affe8ad54-plugins-config-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:10:15.474144 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.474110    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:15.867118 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.867081    2550 generic.go:356] "Generic (PLEG): container finished" podID="69effffe-d680-426a-9518-031affe8ad54" containerID="25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c" exitCode=0
Sep 09 21:10:15.867527 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.867125    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"
Sep 09 21:10:15.867527 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.867183    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc" event={"ID":"69effffe-d680-426a-9518-031affe8ad54","Type":"ContainerDied","Data":"25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c"}
Sep 09 21:10:15.867527 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.867236    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc" event={"ID":"69effffe-d680-426a-9518-031affe8ad54","Type":"ContainerDied","Data":"27fc4f4a0fa4326f4f5c3a0579d6220f150721c5e6be3d61fe82502ac74c1397"}
Sep 09 21:10:15.867527 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.867265    2550 scope.go:117] "RemoveContainer" containerID="25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c"
Sep 09 21:10:15.868686 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.868667    2550 generic.go:356] "Generic (PLEG): container finished" podID="c15e5119-cb46-486a-8b6e-d5a2db44b9ef" containerID="94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72" exitCode=0
Sep 09 21:10:15.868773 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.868718    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"
Sep 09 21:10:15.868773 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.868755    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk" event={"ID":"c15e5119-cb46-486a-8b6e-d5a2db44b9ef","Type":"ContainerDied","Data":"94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72"}
Sep 09 21:10:15.868854 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.868802    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk" event={"ID":"c15e5119-cb46-486a-8b6e-d5a2db44b9ef","Type":"ContainerDied","Data":"8eebc7165fba2d3a7a718cde24a605917466d17fe8b03c4e379b251206893179"}
Sep 09 21:10:15.876071 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.876051    2550 scope.go:117] "RemoveContainer" containerID="25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c"
Sep 09 21:10:15.876361 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:10:15.876341    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c\": container with ID starting with 25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c not found: ID does not exist" containerID="25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c"
Sep 09 21:10:15.876445 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.876367    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c"} err="failed to get container status \"25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c\": rpc error: code = NotFound desc = could not find container \"25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c\": container with ID starting with 25994a36f4ea8c9862d4fc4e1d90dfe7ab3daf6bd5e2a01dc40d83c612ebea0c not found: ID does not exist"
Sep 09 21:10:15.876445 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.876384    2550 scope.go:117] "RemoveContainer" containerID="94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72"
Sep 09 21:10:15.884801 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.884786    2550 scope.go:117] "RemoveContainer" containerID="94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72"
Sep 09 21:10:15.885041 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:10:15.885022    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72\": container with ID starting with 94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72 not found: ID does not exist" containerID="94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72"
Sep 09 21:10:15.885105 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.885049    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72"} err="failed to get container status \"94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72\": rpc error: code = NotFound desc = could not find container \"94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72\": container with ID starting with 94a36841552fda8305cf01b45aebffa082c6949c5b6234194f50a0088157eb72 not found: ID does not exist"
Sep 09 21:10:15.908030 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.907999    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"]
Sep 09 21:10:15.923987 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.923957    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a2fb5a-5fc88b8db-jnmmc"]
Sep 09 21:10:15.946800 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.946775    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"]
Sep 09 21:10:15.951427 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:15.951404    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a2fb5a-68f4b59cdc-tntwk"]
Sep 09 21:10:16.279685 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:16.279657    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69effffe-d680-426a-9518-031affe8ad54" path="/var/lib/kubelet/pods/69effffe-d680-426a-9518-031affe8ad54/volumes"
Sep 09 21:10:16.279942 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:16.279930    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7887e1f3-14d9-4a94-8531-bb59da1aae38" path="/var/lib/kubelet/pods/7887e1f3-14d9-4a94-8531-bb59da1aae38/volumes"
Sep 09 21:10:16.280229 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:16.280217    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c15e5119-cb46-486a-8b6e-d5a2db44b9ef" path="/var/lib/kubelet/pods/c15e5119-cb46-486a-8b6e-d5a2db44b9ef/volumes"
Sep 09 21:10:20.801389 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:20.801357    2550 scope.go:117] "RemoveContainer" containerID="494c9eb865ce1d19c5502489d288d862cb5f1fe35076a6d3367d9e67ddbe4718"
Sep 09 21:10:20.809458 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:20.809439    2550 scope.go:117] "RemoveContainer" containerID="2a4a5f02a63696e724008e8616b3e2f401b07114c831842520543d002c0e1ac5"
Sep 09 21:10:21.468792 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:21.468753    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:21.601530 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:21.601496    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:21.764505 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:21.764409    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:24.270279 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.270243    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-h7v4k"]
Sep 09 21:10:24.270675 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.270449    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-845f9dff7c-h7v4k" podUID="313705ea-49c9-478e-9b48-bf7ca727fb64" containerName="manager" containerID="cri-o://fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e" gracePeriod=30
Sep 09 21:10:24.513265 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.513242    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 21:10:24.602917 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.602843    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-webhook-cert\") pod \"313705ea-49c9-478e-9b48-bf7ca727fb64\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") "
Sep 09 21:10:24.602917 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.602892    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-metrics-certs\") pod \"313705ea-49c9-478e-9b48-bf7ca727fb64\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") "
Sep 09 21:10:24.602917 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.602913    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmpkb\" (UniqueName: \"kubernetes.io/projected/313705ea-49c9-478e-9b48-bf7ca727fb64-kube-api-access-jmpkb\") pod \"313705ea-49c9-478e-9b48-bf7ca727fb64\" (UID: \"313705ea-49c9-478e-9b48-bf7ca727fb64\") "
Sep 09 21:10:24.604892 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.604861    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "313705ea-49c9-478e-9b48-bf7ca727fb64" (UID: "313705ea-49c9-478e-9b48-bf7ca727fb64"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:10:24.605020 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.604938    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "313705ea-49c9-478e-9b48-bf7ca727fb64" (UID: "313705ea-49c9-478e-9b48-bf7ca727fb64"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:10:24.605020 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.604955    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/313705ea-49c9-478e-9b48-bf7ca727fb64-kube-api-access-jmpkb" (OuterVolumeSpecName: "kube-api-access-jmpkb") pod "313705ea-49c9-478e-9b48-bf7ca727fb64" (UID: "313705ea-49c9-478e-9b48-bf7ca727fb64"). InnerVolumeSpecName "kube-api-access-jmpkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:10:24.704110 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.704086    2550 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-metrics-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:10:24.704110 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.704109    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmpkb\" (UniqueName: \"kubernetes.io/projected/313705ea-49c9-478e-9b48-bf7ca727fb64-kube-api-access-jmpkb\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:10:24.704292 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.704119    2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/313705ea-49c9-478e-9b48-bf7ca727fb64-webhook-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:10:24.918780 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.918705    2550 generic.go:356] "Generic (PLEG): container finished" podID="313705ea-49c9-478e-9b48-bf7ca727fb64" containerID="fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e" exitCode=0
Sep 09 21:10:24.918780 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.918749    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-845f9dff7c-h7v4k"
Sep 09 21:10:24.918780 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.918760    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-845f9dff7c-h7v4k" event={"ID":"313705ea-49c9-478e-9b48-bf7ca727fb64","Type":"ContainerDied","Data":"fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e"}
Sep 09 21:10:24.919001 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.918802    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-845f9dff7c-h7v4k" event={"ID":"313705ea-49c9-478e-9b48-bf7ca727fb64","Type":"ContainerDied","Data":"0eb53375ea646ad69f9eea21f385d19cd373fd654310d80a2994dcd86299c4a0"}
Sep 09 21:10:24.919001 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.918822    2550 scope.go:117] "RemoveContainer" containerID="fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e"
Sep 09 21:10:24.928512 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.928498    2550 scope.go:117] "RemoveContainer" containerID="fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e"
Sep 09 21:10:24.928750 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:10:24.928734    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e\": container with ID starting with fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e not found: ID does not exist" containerID="fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e"
Sep 09 21:10:24.928798 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.928759    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e"} err="failed to get container status \"fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e\": rpc error: code = NotFound desc = could not find container \"fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e\": container with ID starting with fd1c5c41f4ae9e003b2185f54ca8bda0ed70b0526eecb16deaa9ba983c19bc9e not found: ID does not exist"
Sep 09 21:10:24.938564 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.938543    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-h7v4k"]
Sep 09 21:10:24.942570 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:24.942548    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-h7v4k"]
Sep 09 21:10:26.278864 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:10:26.278820    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="313705ea-49c9-478e-9b48-bf7ca727fb64" path="/var/lib/kubelet/pods/313705ea-49c9-478e-9b48-bf7ca727fb64/volumes"
Sep 09 21:11:25.723073 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723040    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-845f9dff7c-x8859"]
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723361    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="313705ea-49c9-478e-9b48-bf7ca727fb64" containerName="manager"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723372    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="313705ea-49c9-478e-9b48-bf7ca727fb64" containerName="manager"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723382    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="048a5687-42a4-4b44-beee-db5bd2192a1f" containerName="payload-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723390    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="048a5687-42a4-4b44-beee-db5bd2192a1f" containerName="payload-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723396    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723404    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723411    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3577e639-9ac9-4c14-83db-1b5dac6f67d0" containerName="main"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723416    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3577e639-9ac9-4c14-83db-1b5dac6f67d0" containerName="main"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723423    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c15e5119-cb46-486a-8b6e-d5a2db44b9ef" containerName="payload-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723428    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c15e5119-cb46-486a-8b6e-d5a2db44b9ef" containerName="payload-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723441    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723446    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723451    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef417d3c-c927-4700-9789-05cb639a76d1" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723457    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef417d3c-c927-4700-9789-05cb639a76d1" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723463    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723468    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723474    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723479    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723487    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e1fc10d-f0a0-4142-af35-8e86a9e1e037" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723492    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e1fc10d-f0a0-4142-af35-8e86a9e1e037" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723499    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69effffe-d680-426a-9518-031affe8ad54" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723504    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="69effffe-d680-426a-9518-031affe8ad54" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723552    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723559    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="048a5687-42a4-4b44-beee-db5bd2192a1f" containerName="payload-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723566    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723571    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef417d3c-c927-4700-9789-05cb639a76d1" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723577    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3577e639-9ac9-4c14-83db-1b5dac6f67d0" containerName="main"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723582    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa6a0e5d-1282-46c3-9b48-3faa54c5cbb2" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723587    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723592    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="313705ea-49c9-478e-9b48-bf7ca727fb64" containerName="manager"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723596    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3530dd8c-0f36-4309-bdda-5b481c304d02" containerName="istio-proxy"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723601    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="69effffe-d680-426a-9518-031affe8ad54" containerName="payload-pre-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723607    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c15e5119-cb46-486a-8b6e-d5a2db44b9ef" containerName="payload-processing"
Sep 09 21:11:25.723790 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.723612    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e1fc10d-f0a0-4142-af35-8e86a9e1e037" containerName="payload-pre-processing"
Sep 09 21:11:25.727801 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.727781    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:25.730859 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.730834    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 21:11:25.730991 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.730846    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2pbj7\""
Sep 09 21:11:25.730991 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.730845    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 21:11:25.731116 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.731064    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 21:11:25.737621 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.737597    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-x8859"]
Sep 09 21:11:25.802973 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.802944    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4kgp\" (UniqueName: \"kubernetes.io/projected/fe94005e-8fa4-441e-9792-73cba449c940-kube-api-access-v4kgp\") pod \"maas-controller-845f9dff7c-x8859\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") " pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:25.803128 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.802981    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-webhook-cert\") pod \"maas-controller-845f9dff7c-x8859\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") " pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:25.803128 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.803044    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-metrics-certs\") pod \"maas-controller-845f9dff7c-x8859\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") " pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:25.904288 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.904264    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-metrics-certs\") pod \"maas-controller-845f9dff7c-x8859\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") " pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:25.904404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.904329    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4kgp\" (UniqueName: \"kubernetes.io/projected/fe94005e-8fa4-441e-9792-73cba449c940-kube-api-access-v4kgp\") pod \"maas-controller-845f9dff7c-x8859\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") " pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:25.904404 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.904351    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-webhook-cert\") pod \"maas-controller-845f9dff7c-x8859\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") " pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:25.906704 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.906676    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-metrics-certs\") pod \"maas-controller-845f9dff7c-x8859\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") " pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:25.906814 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.906704    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-webhook-cert\") pod \"maas-controller-845f9dff7c-x8859\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") " pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:25.913329 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:25.913305    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4kgp\" (UniqueName: \"kubernetes.io/projected/fe94005e-8fa4-441e-9792-73cba449c940-kube-api-access-v4kgp\") pod \"maas-controller-845f9dff7c-x8859\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") " pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:26.039002 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:26.038943    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:26.308076 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:26.308047    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-x8859"]
Sep 09 21:11:26.309655 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:11:26.309631    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe94005e_8fa4_441e_9792_73cba449c940.slice/crio-5b4e38255036bf0e5c51cdfc6f0c68f461b462a6b4cd28ba52ac43399b523ec1 WatchSource:0}: Error finding container 5b4e38255036bf0e5c51cdfc6f0c68f461b462a6b4cd28ba52ac43399b523ec1: Status 404 returned error can't find the container with id 5b4e38255036bf0e5c51cdfc6f0c68f461b462a6b4cd28ba52ac43399b523ec1
Sep 09 21:11:26.310946 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:26.310926    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 21:11:27.230928 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:27.230896    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-845f9dff7c-x8859" event={"ID":"fe94005e-8fa4-441e-9792-73cba449c940","Type":"ContainerStarted","Data":"51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85"}
Sep 09 21:11:27.231338 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:27.230955    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-845f9dff7c-x8859" event={"ID":"fe94005e-8fa4-441e-9792-73cba449c940","Type":"ContainerStarted","Data":"5b4e38255036bf0e5c51cdfc6f0c68f461b462a6b4cd28ba52ac43399b523ec1"}
Sep 09 21:11:27.231338 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:27.230984    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:27.248789 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:27.248745    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-845f9dff7c-x8859" podStartSLOduration=1.8056781339999999 podStartE2EDuration="2.248729749s" totalImagesPullingTime="443.051615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:27.246395844 +0000 UTC m=+2347.397068188" watchObservedRunningTime="2026-09-09 21:11:27.248729749 +0000 UTC m=+2347.399402119"
Sep 09 21:11:38.239152 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:38.239117    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:45.568644 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:45.568610    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:47.498932 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.498899    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:47.530519 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.530488    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-x8859"]
Sep 09 21:11:47.530728 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.530701    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-845f9dff7c-x8859" podUID="fe94005e-8fa4-441e-9792-73cba449c940" containerName="manager" containerID="cri-o://51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85" gracePeriod=30
Sep 09 21:11:47.771322 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.771298    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:47.886908 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.886874    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-metrics-certs\") pod \"fe94005e-8fa4-441e-9792-73cba449c940\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") "
Sep 09 21:11:47.887073 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.886939    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4kgp\" (UniqueName: \"kubernetes.io/projected/fe94005e-8fa4-441e-9792-73cba449c940-kube-api-access-v4kgp\") pod \"fe94005e-8fa4-441e-9792-73cba449c940\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") "
Sep 09 21:11:47.887073 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.886959    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-webhook-cert\") pod \"fe94005e-8fa4-441e-9792-73cba449c940\" (UID: \"fe94005e-8fa4-441e-9792-73cba449c940\") "
Sep 09 21:11:47.889090 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.889052    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fe94005e-8fa4-441e-9792-73cba449c940" (UID: "fe94005e-8fa4-441e-9792-73cba449c940"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:11:47.889090 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.889082    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fe94005e-8fa4-441e-9792-73cba449c940" (UID: "fe94005e-8fa4-441e-9792-73cba449c940"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:11:47.889267 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.889150    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe94005e-8fa4-441e-9792-73cba449c940-kube-api-access-v4kgp" (OuterVolumeSpecName: "kube-api-access-v4kgp") pod "fe94005e-8fa4-441e-9792-73cba449c940" (UID: "fe94005e-8fa4-441e-9792-73cba449c940"). InnerVolumeSpecName "kube-api-access-v4kgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:11:47.987519 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.987483    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4kgp\" (UniqueName: \"kubernetes.io/projected/fe94005e-8fa4-441e-9792-73cba449c940-kube-api-access-v4kgp\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:11:47.987519 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.987512    2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-webhook-cert\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:11:47.987519 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:47.987521    2550 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe94005e-8fa4-441e-9792-73cba449c940-metrics-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:11:48.337469 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:48.337433    2550 generic.go:356] "Generic (PLEG): container finished" podID="fe94005e-8fa4-441e-9792-73cba449c940" containerID="51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85" exitCode=0
Sep 09 21:11:48.337635 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:48.337476    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-845f9dff7c-x8859"
Sep 09 21:11:48.337635 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:48.337519    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-845f9dff7c-x8859" event={"ID":"fe94005e-8fa4-441e-9792-73cba449c940","Type":"ContainerDied","Data":"51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85"}
Sep 09 21:11:48.337635 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:48.337566    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-845f9dff7c-x8859" event={"ID":"fe94005e-8fa4-441e-9792-73cba449c940","Type":"ContainerDied","Data":"5b4e38255036bf0e5c51cdfc6f0c68f461b462a6b4cd28ba52ac43399b523ec1"}
Sep 09 21:11:48.337635 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:48.337588    2550 scope.go:117] "RemoveContainer" containerID="51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85"
Sep 09 21:11:48.346287 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:48.346271    2550 scope.go:117] "RemoveContainer" containerID="51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85"
Sep 09 21:11:48.346602 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:11:48.346580    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85\": container with ID starting with 51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85 not found: ID does not exist" containerID="51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85"
Sep 09 21:11:48.346651 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:48.346616    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85"} err="failed to get container status \"51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85\": rpc error: code = NotFound desc = could not find container \"51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85\": container with ID starting with 51f456d9bdc172a2263c8e3bc885a55f50cd0214658af8c87d40a96f47fbab85 not found: ID does not exist"
Sep 09 21:11:48.357755 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:48.357730    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-x8859"]
Sep 09 21:11:48.363843 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:48.363818    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-x8859"]
Sep 09 21:11:49.006660 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.006620    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:49.111490 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.111457    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-845f9dff7c-ft8v9"]
Sep 09 21:11:49.111785 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.111771    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe94005e-8fa4-441e-9792-73cba449c940" containerName="manager"
Sep 09 21:11:49.111840 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.111787    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe94005e-8fa4-441e-9792-73cba449c940" containerName="manager"
Sep 09 21:11:49.111840 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.111802    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy"
Sep 09 21:11:49.111840 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.111807    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eaae6c9-9519-4924-8c3b-a60e5bcaa79e" containerName="istio-proxy"
Sep 09 21:11:49.111955 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.111861    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe94005e-8fa4-441e-9792-73cba449c940" containerName="manager"
Sep 09 21:11:49.116084 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.116066    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.118815 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.118797    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 21:11:49.118943 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.118828    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 21:11:49.118943 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.118872    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 21:11:49.118943 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.118831    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2pbj7\""
Sep 09 21:11:49.127961 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.127938    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-ft8v9"]
Sep 09 21:11:49.199590 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.199558    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2359d829-685c-4f24-8b2a-766e95cd1aae-webhook-cert\") pod \"maas-controller-845f9dff7c-ft8v9\" (UID: \"2359d829-685c-4f24-8b2a-766e95cd1aae\") " pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.199590 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.199592    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2359d829-685c-4f24-8b2a-766e95cd1aae-metrics-certs\") pod \"maas-controller-845f9dff7c-ft8v9\" (UID: \"2359d829-685c-4f24-8b2a-766e95cd1aae\") " pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.199781 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.199665    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdlrd\" (UniqueName: \"kubernetes.io/projected/2359d829-685c-4f24-8b2a-766e95cd1aae-kube-api-access-kdlrd\") pod \"maas-controller-845f9dff7c-ft8v9\" (UID: \"2359d829-685c-4f24-8b2a-766e95cd1aae\") " pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.300224 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.300126    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdlrd\" (UniqueName: \"kubernetes.io/projected/2359d829-685c-4f24-8b2a-766e95cd1aae-kube-api-access-kdlrd\") pod \"maas-controller-845f9dff7c-ft8v9\" (UID: \"2359d829-685c-4f24-8b2a-766e95cd1aae\") " pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.300224 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.300218    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2359d829-685c-4f24-8b2a-766e95cd1aae-webhook-cert\") pod \"maas-controller-845f9dff7c-ft8v9\" (UID: \"2359d829-685c-4f24-8b2a-766e95cd1aae\") " pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.300395 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.300240    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2359d829-685c-4f24-8b2a-766e95cd1aae-metrics-certs\") pod \"maas-controller-845f9dff7c-ft8v9\" (UID: \"2359d829-685c-4f24-8b2a-766e95cd1aae\") " pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.302714 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.302679    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2359d829-685c-4f24-8b2a-766e95cd1aae-webhook-cert\") pod \"maas-controller-845f9dff7c-ft8v9\" (UID: \"2359d829-685c-4f24-8b2a-766e95cd1aae\") " pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.302834 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.302738    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2359d829-685c-4f24-8b2a-766e95cd1aae-metrics-certs\") pod \"maas-controller-845f9dff7c-ft8v9\" (UID: \"2359d829-685c-4f24-8b2a-766e95cd1aae\") " pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.308457 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.308436    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdlrd\" (UniqueName: \"kubernetes.io/projected/2359d829-685c-4f24-8b2a-766e95cd1aae-kube-api-access-kdlrd\") pod \"maas-controller-845f9dff7c-ft8v9\" (UID: \"2359d829-685c-4f24-8b2a-766e95cd1aae\") " pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.426400 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.426369    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:49.702242 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:49.702211    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-845f9dff7c-ft8v9"]
Sep 09 21:11:49.703235 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:11:49.703214    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2359d829_685c_4f24_8b2a_766e95cd1aae.slice/crio-2573f35a213e7ea60d1bc513e6b87bb883f44d436fbbdb90ea03261321e7d092 WatchSource:0}: Error finding container 2573f35a213e7ea60d1bc513e6b87bb883f44d436fbbdb90ea03261321e7d092: Status 404 returned error can't find the container with id 2573f35a213e7ea60d1bc513e6b87bb883f44d436fbbdb90ea03261321e7d092
Sep 09 21:11:50.279255 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:50.279219    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe94005e-8fa4-441e-9792-73cba449c940" path="/var/lib/kubelet/pods/fe94005e-8fa4-441e-9792-73cba449c940/volumes"
Sep 09 21:11:50.350935 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:50.350860    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-845f9dff7c-ft8v9" event={"ID":"2359d829-685c-4f24-8b2a-766e95cd1aae","Type":"ContainerStarted","Data":"5fa3c8777db5f0e3171023e8ac5a66aeb3b2272276ebefcbbd8d3990fe3fe1d8"}
Sep 09 21:11:50.350935 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:50.350911    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-845f9dff7c-ft8v9" event={"ID":"2359d829-685c-4f24-8b2a-766e95cd1aae","Type":"ContainerStarted","Data":"2573f35a213e7ea60d1bc513e6b87bb883f44d436fbbdb90ea03261321e7d092"}
Sep 09 21:11:50.351151 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:50.350978    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:11:50.380009 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:11:50.379958    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-845f9dff7c-ft8v9" podStartSLOduration=0.995384914 podStartE2EDuration="1.37994547s" totalImagesPullingTime="384.560556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:50.378894585 +0000 UTC m=+2370.529566940" watchObservedRunningTime="2026-09-09 21:11:50.37994547 +0000 UTC m=+2370.530617813"
Sep 09 21:12:01.357788 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:01.357750    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-845f9dff7c-ft8v9"
Sep 09 21:12:20.505498 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:20.505467    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 21:12:20.513941 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:20.513920    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bsmvv_d9d5ede3-b0a8-40ba-9856-0d78fab8f78b/ovn-acl-logging/0.log"
Sep 09 21:12:51.830064 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:51.829982    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"]
Sep 09 21:12:51.830577 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:51.830231    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r" podUID="3d05d316-d3e3-40bb-888f-7d821ca1d2f4" containerName="manager" containerID="cri-o://45ae96896dd944bbd57a44b9811dcb716f7d627bb7acaa09bee4784a738f0bd9" gracePeriod=10
Sep 09 21:12:52.666916 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:52.666837    2550 generic.go:356] "Generic (PLEG): container finished" podID="3d05d316-d3e3-40bb-888f-7d821ca1d2f4" containerID="45ae96896dd944bbd57a44b9811dcb716f7d627bb7acaa09bee4784a738f0bd9" exitCode=0
Sep 09 21:12:52.667053 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:52.666912    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r" event={"ID":"3d05d316-d3e3-40bb-888f-7d821ca1d2f4","Type":"ContainerDied","Data":"45ae96896dd944bbd57a44b9811dcb716f7d627bb7acaa09bee4784a738f0bd9"}
Sep 09 21:12:52.667053 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:52.666975    2550 scope.go:117] "RemoveContainer" containerID="485b0880caf3a286bda627eaf289ebb4b8bafdd8e3934899bd7c8ac2bb6737ec"
Sep 09 21:12:52.874043 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:52.874021    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 21:12:53.043473 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.043445    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-extensions-socket-volume\") pod \"3d05d316-d3e3-40bb-888f-7d821ca1d2f4\" (UID: \"3d05d316-d3e3-40bb-888f-7d821ca1d2f4\") "
Sep 09 21:12:53.043634 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.043499    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2dr2\" (UniqueName: \"kubernetes.io/projected/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-kube-api-access-z2dr2\") pod \"3d05d316-d3e3-40bb-888f-7d821ca1d2f4\" (UID: \"3d05d316-d3e3-40bb-888f-7d821ca1d2f4\") "
Sep 09 21:12:53.043848 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.043815    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3d05d316-d3e3-40bb-888f-7d821ca1d2f4" (UID: "3d05d316-d3e3-40bb-888f-7d821ca1d2f4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:12:53.045527 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.045499    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-kube-api-access-z2dr2" (OuterVolumeSpecName: "kube-api-access-z2dr2") pod "3d05d316-d3e3-40bb-888f-7d821ca1d2f4" (UID: "3d05d316-d3e3-40bb-888f-7d821ca1d2f4"). InnerVolumeSpecName "kube-api-access-z2dr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:12:53.144962 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.144931    2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-extensions-socket-volume\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:12:53.144962 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.144955    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2dr2\" (UniqueName: \"kubernetes.io/projected/3d05d316-d3e3-40bb-888f-7d821ca1d2f4-kube-api-access-z2dr2\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:12:53.673365 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.673329    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r" event={"ID":"3d05d316-d3e3-40bb-888f-7d821ca1d2f4","Type":"ContainerDied","Data":"80cfacf5a5d580fbdd34139f2dc940d4bcb1a95ec279f222e84073502b158ec5"}
Sep 09 21:12:53.673543 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.673390    2550 scope.go:117] "RemoveContainer" containerID="45ae96896dd944bbd57a44b9811dcb716f7d627bb7acaa09bee4784a738f0bd9"
Sep 09 21:12:53.673543 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.673349    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"
Sep 09 21:12:53.696772 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.696744    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"]
Sep 09 21:12:53.702218 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:53.702190    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8ch2r"]
Sep 09 21:12:54.279333 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:54.279299    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d05d316-d3e3-40bb-888f-7d821ca1d2f4" path="/var/lib/kubelet/pods/3d05d316-d3e3-40bb-888f-7d821ca1d2f4/volumes"
Sep 09 21:12:59.105904 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:12:59.105871    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:10.816064 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:10.816019    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:31.941696 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:31.941660    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:52.487468 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.487434    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"]
Sep 09 21:13:52.487912 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.487855    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d05d316-d3e3-40bb-888f-7d821ca1d2f4" containerName="manager"
Sep 09 21:13:52.487912 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.487870    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d05d316-d3e3-40bb-888f-7d821ca1d2f4" containerName="manager"
Sep 09 21:13:52.487912 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.487883    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d05d316-d3e3-40bb-888f-7d821ca1d2f4" containerName="manager"
Sep 09 21:13:52.487912 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.487889    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d05d316-d3e3-40bb-888f-7d821ca1d2f4" containerName="manager"
Sep 09 21:13:52.488072 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.487935    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d05d316-d3e3-40bb-888f-7d821ca1d2f4" containerName="manager"
Sep 09 21:13:52.488072 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.487943    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d05d316-d3e3-40bb-888f-7d821ca1d2f4" containerName="manager"
Sep 09 21:13:52.490616 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.490600    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"
Sep 09 21:13:52.492624 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.492604    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-h4wzg\""
Sep 09 21:13:52.502057 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.502036    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"]
Sep 09 21:13:52.622876 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.622849    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clj5v\" (UniqueName: \"kubernetes.io/projected/844eb07e-bc40-4b5a-9030-20e82ec09bb6-kube-api-access-clj5v\") pod \"kuadrant-operator-controller-manager-5769f99d9f-h5fqf\" (UID: \"844eb07e-bc40-4b5a-9030-20e82ec09bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"
Sep 09 21:13:52.623039 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.622928    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/844eb07e-bc40-4b5a-9030-20e82ec09bb6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-h5fqf\" (UID: \"844eb07e-bc40-4b5a-9030-20e82ec09bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"
Sep 09 21:13:52.723680 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.723635    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/844eb07e-bc40-4b5a-9030-20e82ec09bb6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-h5fqf\" (UID: \"844eb07e-bc40-4b5a-9030-20e82ec09bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"
Sep 09 21:13:52.723852 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.723691    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clj5v\" (UniqueName: \"kubernetes.io/projected/844eb07e-bc40-4b5a-9030-20e82ec09bb6-kube-api-access-clj5v\") pod \"kuadrant-operator-controller-manager-5769f99d9f-h5fqf\" (UID: \"844eb07e-bc40-4b5a-9030-20e82ec09bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"
Sep 09 21:13:52.724037 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.724014    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/844eb07e-bc40-4b5a-9030-20e82ec09bb6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-h5fqf\" (UID: \"844eb07e-bc40-4b5a-9030-20e82ec09bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"
Sep 09 21:13:52.732583 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.732556    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clj5v\" (UniqueName: \"kubernetes.io/projected/844eb07e-bc40-4b5a-9030-20e82ec09bb6-kube-api-access-clj5v\") pod \"kuadrant-operator-controller-manager-5769f99d9f-h5fqf\" (UID: \"844eb07e-bc40-4b5a-9030-20e82ec09bb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"
Sep 09 21:13:52.800409 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:52.800325    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"
Sep 09 21:13:53.092679 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:53.092648    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"]
Sep 09 21:13:53.967400 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:53.967359    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf" event={"ID":"844eb07e-bc40-4b5a-9030-20e82ec09bb6","Type":"ContainerStarted","Data":"25d1de138ed4207d4605ea6f903c5955c4fd8bcd898cdb63965991fc802f8b97"}
Sep 09 21:13:53.967784 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:53.967426    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"
Sep 09 21:13:53.967784 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:13:53.967448    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf" event={"ID":"844eb07e-bc40-4b5a-9030-20e82ec09bb6","Type":"ContainerStarted","Data":"bb0c14257c525cb869ce4ce1a87b939ff936dae17c50b8baf2e9af4f7bb866af"}
Sep 09 21:14:04.975088 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:04.975057    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf"
Sep 09 21:14:04.994724 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:04.994675    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-h5fqf" podStartSLOduration=12.994660707 podStartE2EDuration="12.994660707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:54.001952569 +0000 UTC m=+2494.152624913" watchObservedRunningTime="2026-09-09 21:14:04.994660707 +0000 UTC m=+2505.145333049"
Sep 09 21:14:08.529455 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:08.529415    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:08.593710 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:08.593674    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:23.821740 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:23.821705    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:25.813700 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.813665    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"]
Sep 09 21:14:25.817021 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.817003    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:25.819631 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.819613    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-91468989-kserve-self-signed-certs\""
Sep 09 21:14:25.831634 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.831610    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"]
Sep 09 21:14:25.886415 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.886389    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:25.906410 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.906386    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-home\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:25.906499 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.906418    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/630cb347-6b94-4c5a-bdcf-2cc08867816b-tls-certs\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:25.906499 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.906459    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-model-cache\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:25.906574 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.906498    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-dshm\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:25.906574 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.906517    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdxtf\" (UniqueName: \"kubernetes.io/projected/630cb347-6b94-4c5a-bdcf-2cc08867816b-kube-api-access-bdxtf\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:25.906639 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:25.906600    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-tmp-dir\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.007214 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.007179    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-home\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.007396 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.007224    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/630cb347-6b94-4c5a-bdcf-2cc08867816b-tls-certs\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.007396 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.007283    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-model-cache\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.007396 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.007321    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-dshm\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.007396 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.007350    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdxtf\" (UniqueName: \"kubernetes.io/projected/630cb347-6b94-4c5a-bdcf-2cc08867816b-kube-api-access-bdxtf\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.007396 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.007401    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-tmp-dir\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.007697 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.007643    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-home\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.007807 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.007725    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-tmp-dir\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.007807 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.007766    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-model-cache\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.009634 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.009604    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-dshm\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.009771 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.009745    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/630cb347-6b94-4c5a-bdcf-2cc08867816b-tls-certs\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.017580 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.017562    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdxtf\" (UniqueName: \"kubernetes.io/projected/630cb347-6b94-4c5a-bdcf-2cc08867816b-kube-api-access-bdxtf\") pod \"e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") " pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.135402 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.135308    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:26.411585 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:26.411554    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"]
Sep 09 21:14:26.413314 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:14:26.413289    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod630cb347_6b94_4c5a_bdcf_2cc08867816b.slice/crio-abf7d9bd9de4f92cb51fcf1b1c5513ab313cd54b28255655171473a88bfd8155 WatchSource:0}: Error finding container abf7d9bd9de4f92cb51fcf1b1c5513ab313cd54b28255655171473a88bfd8155: Status 404 returned error can't find the container with id abf7d9bd9de4f92cb51fcf1b1c5513ab313cd54b28255655171473a88bfd8155
Sep 09 21:14:27.133038 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:27.132997    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp" event={"ID":"630cb347-6b94-4c5a-bdcf-2cc08867816b","Type":"ContainerStarted","Data":"126a11c520181e4c00aaec02d3c2e789120387040ac7bafb01730d99cc904096"}
Sep 09 21:14:27.133038 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:27.133052    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp" event={"ID":"630cb347-6b94-4c5a-bdcf-2cc08867816b","Type":"ContainerStarted","Data":"abf7d9bd9de4f92cb51fcf1b1c5513ab313cd54b28255655171473a88bfd8155"}
Sep 09 21:14:27.153198 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:27.153137    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp" podStartSLOduration=2.15311963 podStartE2EDuration="2.15311963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:27.151584755 +0000 UTC m=+2527.302257095" watchObservedRunningTime="2026-09-09 21:14:27.15311963 +0000 UTC m=+2527.303791972"
Sep 09 21:14:36.135814 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:36.135784    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:36.136331 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:36.136062    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:36.148402 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:36.148381    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:36.191547 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:36.191524    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:14:52.442107 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:14:52.442069    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:08.529403 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:08.529350    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:13.251320 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:13.251282    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"]
Sep 09 21:15:13.276287 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:13.276259    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:14.698379 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:14.698345    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:28.271584 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.271521    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp" podUID="630cb347-6b94-4c5a-bdcf-2cc08867816b" containerName="main" containerID="cri-o://126a11c520181e4c00aaec02d3c2e789120387040ac7bafb01730d99cc904096" gracePeriod=45
Sep 09 21:15:28.452293 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.452265    2550 generic.go:356] "Generic (PLEG): container finished" podID="630cb347-6b94-4c5a-bdcf-2cc08867816b" containerID="126a11c520181e4c00aaec02d3c2e789120387040ac7bafb01730d99cc904096" exitCode=0
Sep 09 21:15:28.452437 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.452343    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp" event={"ID":"630cb347-6b94-4c5a-bdcf-2cc08867816b","Type":"ContainerDied","Data":"126a11c520181e4c00aaec02d3c2e789120387040ac7bafb01730d99cc904096"}
Sep 09 21:15:28.520783 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.520759    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:15:28.637810 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.637721    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-tmp-dir\") pod \"630cb347-6b94-4c5a-bdcf-2cc08867816b\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") "
Sep 09 21:15:28.637810 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.637756    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-dshm\") pod \"630cb347-6b94-4c5a-bdcf-2cc08867816b\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") "
Sep 09 21:15:28.637810 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.637783    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-home\") pod \"630cb347-6b94-4c5a-bdcf-2cc08867816b\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") "
Sep 09 21:15:28.637810 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.637810    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdxtf\" (UniqueName: \"kubernetes.io/projected/630cb347-6b94-4c5a-bdcf-2cc08867816b-kube-api-access-bdxtf\") pod \"630cb347-6b94-4c5a-bdcf-2cc08867816b\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") "
Sep 09 21:15:28.638143 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.637868    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-model-cache\") pod \"630cb347-6b94-4c5a-bdcf-2cc08867816b\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") "
Sep 09 21:15:28.638143 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.637905    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/630cb347-6b94-4c5a-bdcf-2cc08867816b-tls-certs\") pod \"630cb347-6b94-4c5a-bdcf-2cc08867816b\" (UID: \"630cb347-6b94-4c5a-bdcf-2cc08867816b\") "
Sep 09 21:15:28.638143 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.638063    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "630cb347-6b94-4c5a-bdcf-2cc08867816b" (UID: "630cb347-6b94-4c5a-bdcf-2cc08867816b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:15:28.638329 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.638209    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-tmp-dir\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:15:28.638329 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.638210    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-home" (OuterVolumeSpecName: "home") pod "630cb347-6b94-4c5a-bdcf-2cc08867816b" (UID: "630cb347-6b94-4c5a-bdcf-2cc08867816b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:15:28.638329 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.638239    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-model-cache" (OuterVolumeSpecName: "model-cache") pod "630cb347-6b94-4c5a-bdcf-2cc08867816b" (UID: "630cb347-6b94-4c5a-bdcf-2cc08867816b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:15:28.640014 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.639989    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-dshm" (OuterVolumeSpecName: "dshm") pod "630cb347-6b94-4c5a-bdcf-2cc08867816b" (UID: "630cb347-6b94-4c5a-bdcf-2cc08867816b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:15:28.640113 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.640084    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/630cb347-6b94-4c5a-bdcf-2cc08867816b-kube-api-access-bdxtf" (OuterVolumeSpecName: "kube-api-access-bdxtf") pod "630cb347-6b94-4c5a-bdcf-2cc08867816b" (UID: "630cb347-6b94-4c5a-bdcf-2cc08867816b"). InnerVolumeSpecName "kube-api-access-bdxtf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:15:28.640210 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.640126    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/630cb347-6b94-4c5a-bdcf-2cc08867816b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "630cb347-6b94-4c5a-bdcf-2cc08867816b" (UID: "630cb347-6b94-4c5a-bdcf-2cc08867816b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:15:28.739479 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.739443    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/630cb347-6b94-4c5a-bdcf-2cc08867816b-tls-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:15:28.739479 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.739472    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-dshm\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:15:28.739479 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.739481    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-home\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:15:28.739479 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.739489    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdxtf\" (UniqueName: \"kubernetes.io/projected/630cb347-6b94-4c5a-bdcf-2cc08867816b-kube-api-access-bdxtf\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:15:28.739753 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:28.739499    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/630cb347-6b94-4c5a-bdcf-2cc08867816b-model-cache\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:15:29.458692 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:29.458661    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp" event={"ID":"630cb347-6b94-4c5a-bdcf-2cc08867816b","Type":"ContainerDied","Data":"abf7d9bd9de4f92cb51fcf1b1c5513ab313cd54b28255655171473a88bfd8155"}
Sep 09 21:15:29.459098 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:29.458703    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"
Sep 09 21:15:29.459098 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:29.458711    2550 scope.go:117] "RemoveContainer" containerID="126a11c520181e4c00aaec02d3c2e789120387040ac7bafb01730d99cc904096"
Sep 09 21:15:29.483765 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:29.483737    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"]
Sep 09 21:15:29.488618 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:29.488598    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-91468989-kserve-784f7d96f9-kr2sp"]
Sep 09 21:15:30.279518 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:30.279484    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="630cb347-6b94-4c5a-bdcf-2cc08867816b" path="/var/lib/kubelet/pods/630cb347-6b94-4c5a-bdcf-2cc08867816b/volumes"
Sep 09 21:15:31.706891 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.706858    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"]
Sep 09 21:15:31.707262 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.707210    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="630cb347-6b94-4c5a-bdcf-2cc08867816b" containerName="main"
Sep 09 21:15:31.707262 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.707222    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="630cb347-6b94-4c5a-bdcf-2cc08867816b" containerName="main"
Sep 09 21:15:31.707343 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.707286    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="630cb347-6b94-4c5a-bdcf-2cc08867816b" containerName="main"
Sep 09 21:15:31.711603 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.711579    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.724841 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.724813    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"]
Sep 09 21:15:31.764559 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.764528    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-openshift-service-ca\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.764710 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.764569    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tj44\" (UniqueName: \"kubernetes.io/projected/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-kube-api-access-8tj44\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.764710 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.764588    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-maas-api-tls\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.764710 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.764680    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-metrics-certs\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.865638 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.865598    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tj44\" (UniqueName: \"kubernetes.io/projected/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-kube-api-access-8tj44\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.865638 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.865636    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-maas-api-tls\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.865872 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.865681    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-metrics-certs\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.865872 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.865764    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-openshift-service-ca\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.866481 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.866455    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-openshift-service-ca\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.868136 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.868112    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-metrics-certs\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.868293 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.868185    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-maas-api-tls\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:31.873058 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:31.873033    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tj44\" (UniqueName: \"kubernetes.io/projected/144b057e-4dcc-4ff9-9b83-731cedb4ad2c-kube-api-access-8tj44\") pod \"maas-api-57c49c564c-42jgn\" (UID: \"144b057e-4dcc-4ff9-9b83-731cedb4ad2c\") " pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:32.023646 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:32.023601    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:32.305027 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:32.305001    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"]
Sep 09 21:15:32.307105 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:15:32.307079    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod144b057e_4dcc_4ff9_9b83_731cedb4ad2c.slice/crio-d5317b9de2b8b7edfab267cdca4e2cdfcccc0d38c9b919d93a5985f131c9abbf WatchSource:0}: Error finding container d5317b9de2b8b7edfab267cdca4e2cdfcccc0d38c9b919d93a5985f131c9abbf: Status 404 returned error can't find the container with id d5317b9de2b8b7edfab267cdca4e2cdfcccc0d38c9b919d93a5985f131c9abbf
Sep 09 21:15:32.477886 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:32.477812    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn" event={"ID":"144b057e-4dcc-4ff9-9b83-731cedb4ad2c","Type":"ContainerStarted","Data":"d5317b9de2b8b7edfab267cdca4e2cdfcccc0d38c9b919d93a5985f131c9abbf"}
Sep 09 21:15:33.484734 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:33.484695    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn" event={"ID":"144b057e-4dcc-4ff9-9b83-731cedb4ad2c","Type":"ContainerStarted","Data":"c0405126454d03ed4920ad533edc4910a908fdc2d97a6fd628c90db1fca9976a"}
Sep 09 21:15:33.485109 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:33.484763    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:39.495559 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:39.495525    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn"
Sep 09 21:15:39.512465 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:39.512416    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-57c49c564c-42jgn" podStartSLOduration=7.93981874 podStartE2EDuration="8.512400418s" totalImagesPullingTime="572.581678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:33.507869138 +0000 UTC m=+2593.658541482" watchObservedRunningTime="2026-09-09 21:15:39.512400418 +0000 UTC m=+2599.663072761"
Sep 09 21:15:39.561355 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:39.561318    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"]
Sep 09 21:15:39.561599 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:39.561567    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4" podUID="caf2c30b-351e-42ef-b1f1-602f8dee730b" containerName="maas-api" containerID="cri-o://1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818" gracePeriod=30
Sep 09 21:15:40.321641 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:40.321603    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4" podUID="caf2c30b-351e-42ef-b1f1-602f8dee730b" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.45:8443/health\": dial tcp 10.132.0.45:8443: connect: connection refused"
Sep 09 21:15:40.806762 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:40.806735    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 21:15:40.943889 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:40.943801    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgqtk\" (UniqueName: \"kubernetes.io/projected/caf2c30b-351e-42ef-b1f1-602f8dee730b-kube-api-access-pgqtk\") pod \"caf2c30b-351e-42ef-b1f1-602f8dee730b\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") "
Sep 09 21:15:40.943889 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:40.943878    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls\") pod \"caf2c30b-351e-42ef-b1f1-602f8dee730b\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") "
Sep 09 21:15:40.944129 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:40.943901    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-metrics-certs\") pod \"caf2c30b-351e-42ef-b1f1-602f8dee730b\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") "
Sep 09 21:15:40.944129 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:40.943932    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf2c30b-351e-42ef-b1f1-602f8dee730b-openshift-service-ca\") pod \"caf2c30b-351e-42ef-b1f1-602f8dee730b\" (UID: \"caf2c30b-351e-42ef-b1f1-602f8dee730b\") "
Sep 09 21:15:40.944367 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:40.944338    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf2c30b-351e-42ef-b1f1-602f8dee730b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "caf2c30b-351e-42ef-b1f1-602f8dee730b" (UID: "caf2c30b-351e-42ef-b1f1-602f8dee730b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:15:40.946119 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:40.946089    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "caf2c30b-351e-42ef-b1f1-602f8dee730b" (UID: "caf2c30b-351e-42ef-b1f1-602f8dee730b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:15:40.946284 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:40.946119    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf2c30b-351e-42ef-b1f1-602f8dee730b-kube-api-access-pgqtk" (OuterVolumeSpecName: "kube-api-access-pgqtk") pod "caf2c30b-351e-42ef-b1f1-602f8dee730b" (UID: "caf2c30b-351e-42ef-b1f1-602f8dee730b"). InnerVolumeSpecName "kube-api-access-pgqtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:15:40.946284 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:40.946247    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "caf2c30b-351e-42ef-b1f1-602f8dee730b" (UID: "caf2c30b-351e-42ef-b1f1-602f8dee730b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:15:41.044915 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.044882    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgqtk\" (UniqueName: \"kubernetes.io/projected/caf2c30b-351e-42ef-b1f1-602f8dee730b-kube-api-access-pgqtk\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:15:41.044915 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.044916    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-maas-api-tls\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:15:41.045121 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.044929    2550 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caf2c30b-351e-42ef-b1f1-602f8dee730b-metrics-certs\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:15:41.045121 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.044942    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf2c30b-351e-42ef-b1f1-602f8dee730b-openshift-service-ca\") on node \"ip-10-0-140-126.ec2.internal\" DevicePath \"\""
Sep 09 21:15:41.527818 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.527783    2550 generic.go:356] "Generic (PLEG): container finished" podID="caf2c30b-351e-42ef-b1f1-602f8dee730b" containerID="1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818" exitCode=0
Sep 09 21:15:41.527818 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.527824    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4" event={"ID":"caf2c30b-351e-42ef-b1f1-602f8dee730b","Type":"ContainerDied","Data":"1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818"}
Sep 09 21:15:41.528045 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.527830    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"
Sep 09 21:15:41.528045 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.527863    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4" event={"ID":"caf2c30b-351e-42ef-b1f1-602f8dee730b","Type":"ContainerDied","Data":"dffab14e8c2ca07a21f2e736920c690cbc336481aa551ff2d952d4baf0f98029"}
Sep 09 21:15:41.528045 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.527885    2550 scope.go:117] "RemoveContainer" containerID="1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818"
Sep 09 21:15:41.537132 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.537111    2550 scope.go:117] "RemoveContainer" containerID="1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818"
Sep 09 21:15:41.537456 ip-10-0-140-126 kubenswrapper[2550]: E0909 21:15:41.537430    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818\": container with ID starting with 1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818 not found: ID does not exist" containerID="1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818"
Sep 09 21:15:41.537516 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.537472    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818"} err="failed to get container status \"1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818\": rpc error: code = NotFound desc = could not find container \"1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818\": container with ID starting with 1754745e91475a8ed522ac51b725faf18ffc3ab1552cce301e718622f42fd818 not found: ID does not exist"
Sep 09 21:15:41.549068 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.549044    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"]
Sep 09 21:15:41.552573 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:41.552549    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fc5cc7c7f-2kvd4"]
Sep 09 21:15:42.279898 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:42.279868    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caf2c30b-351e-42ef-b1f1-602f8dee730b" path="/var/lib/kubelet/pods/caf2c30b-351e-42ef-b1f1-602f8dee730b/volumes"
Sep 09 21:15:52.230021 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:52.229938    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:52.343098 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:15:52.343060    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:01.213311 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:01.213281    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d6bdbd8d7-rjdjc_8ff09826-3532-4a67-818a-7ff83588fc37/authorino/0.log"
Sep 09 21:16:06.280502 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:06.280467    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9tm2k_ebf3005c-ac3e-4904-a0ea-33326e1979f0/manager/0.log"
Sep 09 21:16:06.395468 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:06.395435    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dscmd_61c527f5-57d1-404a-bffa-bcbb1433d443/manager/0.log"
Sep 09 21:16:06.511669 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:06.511636    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-845f9dff7c-ft8v9_2359d829-685c-4f24-8b2a-766e95cd1aae/manager/0.log"
Sep 09 21:16:06.861716 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:06.861689    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859cf57674-5fdk2_394ec89c-e336-4d95-a554-5e7080fe9862/copy-manifests/0.log"
Sep 09 21:16:06.869325 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:06.869303    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859cf57674-5fdk2_394ec89c-e336-4d95-a554-5e7080fe9862/manager/0.log"
Sep 09 21:16:07.441533 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:07.441502    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-57c49c564c-42jgn_144b057e-4dcc-4ff9-9b83-731cedb4ad2c/maas-api/0.log"
Sep 09 21:16:07.748148 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:07.748115    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29816460-sfl4j_b6aabd45-004c-4f68-9991-b96e0a79c819/cleanup/0.log"
Sep 09 21:16:08.680951 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:08.680917    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d6bdbd8d7-rjdjc_8ff09826-3532-4a67-818a-7ff83588fc37/authorino/0.log"
Sep 09 21:16:08.791285 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:08.791240    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-wgr2t_a5138784-3a4c-4e84-b161-bb8df8eaceef/manager/0.log"
Sep 09 21:16:08.905553 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:08.905520    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-kjh5x_62fc8f6b-a719-4dde-968f-e8ea792dfe3b/manager/0.log"
Sep 09 21:16:09.122655 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:09.122625    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-h5fqf_844eb07e-bc40-4b5a-9030-20e82ec09bb6/manager/0.log"
Sep 09 21:16:09.360432 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:09.360400    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-5np4r_f5076a56-0459-463a-a676-aba0f9fa56f2/manager/0.log"
Sep 09 21:16:09.688747 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:09.688715    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c24thp_b5d19ecc-2831-40b5-a6fa-9f444be087f8/istio-proxy/0.log"
Sep 09 21:16:09.912301 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:09.912270    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65d657c8d5-cww2l_296ba124-bae7-497c-bf5b-1331e85f4b77/kube-auth-proxy/0.log"
Sep 09 21:16:10.131957 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:10.131929    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jgz67_3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c/istio-proxy/1.log"
Sep 09 21:16:10.364891 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:10.364860    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8n4mh_85871292-25fc-4b5d-ad75-c0df6e970f08/payload-processing/0.log"
Sep 09 21:16:11.108703 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:11.108671    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5f27f_e12aa589-8f4c-4f5f-ac18-aef7f100ffec/main/0.log"
Sep 09 21:16:16.898040 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.898009    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8prvh/must-gather-fx6cr"]
Sep 09 21:16:16.898426 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.898351    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf2c30b-351e-42ef-b1f1-602f8dee730b" containerName="maas-api"
Sep 09 21:16:16.898426 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.898363    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf2c30b-351e-42ef-b1f1-602f8dee730b" containerName="maas-api"
Sep 09 21:16:16.898499 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.898427    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="caf2c30b-351e-42ef-b1f1-602f8dee730b" containerName="maas-api"
Sep 09 21:16:16.901723 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.901706    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8prvh/must-gather-fx6cr"
Sep 09 21:16:16.903749 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.903728    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8prvh\"/\"default-dockercfg-lr2bl\""
Sep 09 21:16:16.904142 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.904126    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8prvh\"/\"openshift-service-ca.crt\""
Sep 09 21:16:16.904283 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.904266    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8prvh\"/\"kube-root-ca.crt\""
Sep 09 21:16:16.910361 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.910337    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8prvh/must-gather-fx6cr"]
Sep 09 21:16:16.921989 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.921954    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:16.954849 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.954824    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0262d453-a574-4e77-9829-b3ebdec16ce6-must-gather-output\") pod \"must-gather-fx6cr\" (UID: \"0262d453-a574-4e77-9829-b3ebdec16ce6\") " pod="openshift-must-gather-8prvh/must-gather-fx6cr"
Sep 09 21:16:16.954962 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:16.954871    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nkjg\" (UniqueName: \"kubernetes.io/projected/0262d453-a574-4e77-9829-b3ebdec16ce6-kube-api-access-5nkjg\") pod \"must-gather-fx6cr\" (UID: \"0262d453-a574-4e77-9829-b3ebdec16ce6\") " pod="openshift-must-gather-8prvh/must-gather-fx6cr"
Sep 09 21:16:17.028275 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:17.028243    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:17.056017 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:17.055990    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0262d453-a574-4e77-9829-b3ebdec16ce6-must-gather-output\") pod \"must-gather-fx6cr\" (UID: \"0262d453-a574-4e77-9829-b3ebdec16ce6\") " pod="openshift-must-gather-8prvh/must-gather-fx6cr"
Sep 09 21:16:17.056143 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:17.056038    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nkjg\" (UniqueName: \"kubernetes.io/projected/0262d453-a574-4e77-9829-b3ebdec16ce6-kube-api-access-5nkjg\") pod \"must-gather-fx6cr\" (UID: \"0262d453-a574-4e77-9829-b3ebdec16ce6\") " pod="openshift-must-gather-8prvh/must-gather-fx6cr"
Sep 09 21:16:17.056371 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:17.056345    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0262d453-a574-4e77-9829-b3ebdec16ce6-must-gather-output\") pod \"must-gather-fx6cr\" (UID: \"0262d453-a574-4e77-9829-b3ebdec16ce6\") " pod="openshift-must-gather-8prvh/must-gather-fx6cr"
Sep 09 21:16:17.070348 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:17.070326    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nkjg\" (UniqueName: \"kubernetes.io/projected/0262d453-a574-4e77-9829-b3ebdec16ce6-kube-api-access-5nkjg\") pod \"must-gather-fx6cr\" (UID: \"0262d453-a574-4e77-9829-b3ebdec16ce6\") " pod="openshift-must-gather-8prvh/must-gather-fx6cr"
Sep 09 21:16:17.211968 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:17.211937    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8prvh/must-gather-fx6cr"
Sep 09 21:16:17.483087 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:17.483062    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8prvh/must-gather-fx6cr"]
Sep 09 21:16:17.484499 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:16:17.484475    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0262d453_a574_4e77_9829_b3ebdec16ce6.slice/crio-9d4d6a895b876adae57a3c99a7efad4b1c61ae591d88e2f3e13f9159f0cc6eda WatchSource:0}: Error finding container 9d4d6a895b876adae57a3c99a7efad4b1c61ae591d88e2f3e13f9159f0cc6eda: Status 404 returned error can't find the container with id 9d4d6a895b876adae57a3c99a7efad4b1c61ae591d88e2f3e13f9159f0cc6eda
Sep 09 21:16:17.717526 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:17.717494    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8prvh/must-gather-fx6cr" event={"ID":"0262d453-a574-4e77-9829-b3ebdec16ce6","Type":"ContainerStarted","Data":"9d4d6a895b876adae57a3c99a7efad4b1c61ae591d88e2f3e13f9159f0cc6eda"}
Sep 09 21:16:18.726201 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:18.726144    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8prvh/must-gather-fx6cr" event={"ID":"0262d453-a574-4e77-9829-b3ebdec16ce6","Type":"ContainerStarted","Data":"73762e5f4c6831cf7a35058a4aa8e1c5c7053ec7dee7bd8bb14fa9c05158c65a"}
Sep 09 21:16:18.726686 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:18.726219    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8prvh/must-gather-fx6cr" event={"ID":"0262d453-a574-4e77-9829-b3ebdec16ce6","Type":"ContainerStarted","Data":"f033e722adfb38fa2fcd2771087a6d5f146a1764eac865942a0870b1bb635f0e"}
Sep 09 21:16:18.741430 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:18.741380    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8prvh/must-gather-fx6cr" podStartSLOduration=1.975165413 podStartE2EDuration="2.741363514s" totalImagesPullingTime="766.198101ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:18.740060773 +0000 UTC m=+2638.890733115" watchObservedRunningTime="2026-09-09 21:16:18.741363514 +0000 UTC m=+2638.892035856"
Sep 09 21:16:19.923041 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:19.922999    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z64xx_eca0688d-9168-4fe9-8ee9-4db243c38e7d/global-pull-secret-syncer/0.log"
Sep 09 21:16:19.988277 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:19.988237    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b2m24_eeb00f2d-fb48-43d3-aee3-83a1fe8881c9/konnectivity-agent/0.log"
Sep 09 21:16:20.180446 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:20.180366    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-126.ec2.internal_dbc80b7331391404881e6f559b693033/haproxy/0.log"
Sep 09 21:16:24.833015 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:24.831750    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d6bdbd8d7-rjdjc_8ff09826-3532-4a67-818a-7ff83588fc37/authorino/0.log"
Sep 09 21:16:24.876380 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:24.876343    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-wgr2t_a5138784-3a4c-4e84-b161-bb8df8eaceef/manager/0.log"
Sep 09 21:16:24.908760 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:24.908728    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-kjh5x_62fc8f6b-a719-4dde-968f-e8ea792dfe3b/manager/0.log"
Sep 09 21:16:24.983083 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:24.983053    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-h5fqf_844eb07e-bc40-4b5a-9030-20e82ec09bb6/manager/0.log"
Sep 09 21:16:25.085673 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:25.085583    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-5np4r_f5076a56-0459-463a-a676-aba0f9fa56f2/manager/0.log"
Sep 09 21:16:26.781281 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:26.781257    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-76nqd_52aa8ec2-fa0c-47a8-be52-2fc776bc2983/node-exporter/0.log"
Sep 09 21:16:26.841973 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:26.841905    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-76nqd_52aa8ec2-fa0c-47a8-be52-2fc776bc2983/kube-rbac-proxy/0.log"
Sep 09 21:16:26.906824 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:26.906797    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-76nqd_52aa8ec2-fa0c-47a8-be52-2fc776bc2983/init-textfile/0.log"
Sep 09 21:16:30.132219 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.132187    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"]
Sep 09 21:16:30.140147 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.140127    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.145120 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.145090    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"]
Sep 09 21:16:30.198293 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.198259    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-lib-modules\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.198488 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.198302    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-sys\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.198488 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.198340    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-podres\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.198488 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.198426    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-proc\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.198488 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.198465    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whjs7\" (UniqueName: \"kubernetes.io/projected/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-kube-api-access-whjs7\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.299398 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.299353    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-proc\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.299580 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.299406    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whjs7\" (UniqueName: \"kubernetes.io/projected/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-kube-api-access-whjs7\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.299580 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.299495    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-lib-modules\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.299580 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.299524    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-sys\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.299580 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.299560    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-podres\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.299799 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.299723    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-podres\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.299799 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.299781    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-proc\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.300205 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.300181    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-lib-modules\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.300297 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.300244    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-sys\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.308743 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.308711    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whjs7\" (UniqueName: \"kubernetes.io/projected/33c94bcf-d5a2-45c6-be78-583b7f4de9f1-kube-api-access-whjs7\") pod \"perf-node-gather-daemonset-sgfwp\" (UID: \"33c94bcf-d5a2-45c6-be78-583b7f4de9f1\") " pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.453827 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.453748    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:30.752627 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.752596    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"]
Sep 09 21:16:30.754701 ip-10-0-140-126 kubenswrapper[2550]: W0909 21:16:30.754660    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod33c94bcf_d5a2_45c6_be78_583b7f4de9f1.slice/crio-370f44e4d13532321a602756d78d4e53457cb3397c8d64adc20e69d05837505d WatchSource:0}: Error finding container 370f44e4d13532321a602756d78d4e53457cb3397c8d64adc20e69d05837505d: Status 404 returned error can't find the container with id 370f44e4d13532321a602756d78d4e53457cb3397c8d64adc20e69d05837505d
Sep 09 21:16:30.828735 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:30.828692    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp" event={"ID":"33c94bcf-d5a2-45c6-be78-583b7f4de9f1","Type":"ContainerStarted","Data":"370f44e4d13532321a602756d78d4e53457cb3397c8d64adc20e69d05837505d"}
Sep 09 21:16:31.838487 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:31.838444    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp" event={"ID":"33c94bcf-d5a2-45c6-be78-583b7f4de9f1","Type":"ContainerStarted","Data":"5c321781881c7db15d5a2ce94ccf7e9471b22e2a041b3b6c752cc393de0e13fb"}
Sep 09 21:16:31.839322 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:31.839297    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:31.856801 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:31.856756    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp" podStartSLOduration=1.856740704 podStartE2EDuration="1.856740704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:31.854187363 +0000 UTC m=+2652.004859701" watchObservedRunningTime="2026-09-09 21:16:31.856740704 +0000 UTC m=+2652.007413046"
Sep 09 21:16:32.362511 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:32.362475    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2jfmw_ef22136f-b71c-48bf-83ee-01d0881f8866/dns/0.log"
Sep 09 21:16:32.387652 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:32.387620    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2jfmw_ef22136f-b71c-48bf-83ee-01d0881f8866/kube-rbac-proxy/0.log"
Sep 09 21:16:32.520745 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:32.520715    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b5k8p_8f54dbf2-1cc5-4c0e-91c0-8dc53551965e/dns-node-resolver/0.log"
Sep 09 21:16:33.135943 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:33.135912    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k6x8t_fbfc5682-d88a-45cf-b4c2-aac06e054300/node-ca/0.log"
Sep 09 21:16:34.139729 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:34.139695    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c24thp_b5d19ecc-2831-40b5-a6fa-9f444be087f8/istio-proxy/0.log"
Sep 09 21:16:34.714385 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:34.714353    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65d657c8d5-cww2l_296ba124-bae7-497c-bf5b-1331e85f4b77/kube-auth-proxy/0.log"
Sep 09 21:16:34.775295 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:34.775264    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jgz67_3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c/istio-proxy/0.log"
Sep 09 21:16:34.780740 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:34.780715    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jgz67_3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c/istio-proxy/1.log"
Sep 09 21:16:34.864530 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:34.864502    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8n4mh_85871292-25fc-4b5d-ad75-c0df6e970f08/payload-processing/0.log"
Sep 09 21:16:35.778087 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:35.778058    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-krxjb_38f31112-a172-4677-be02-544c74a0e0aa/serve-healthcheck-canary/0.log"
Sep 09 21:16:36.373027 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:36.372999    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5ltp2_9c30feed-f1d0-44a7-9764-e0a7f4183d6e/kube-rbac-proxy/0.log"
Sep 09 21:16:36.396816 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:36.396788    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5ltp2_9c30feed-f1d0-44a7-9764-e0a7f4183d6e/exporter/0.log"
Sep 09 21:16:36.422185 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:36.422141    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5ltp2_9c30feed-f1d0-44a7-9764-e0a7f4183d6e/extractor/0.log"
Sep 09 21:16:38.769319 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:38.769286    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9tm2k_ebf3005c-ac3e-4904-a0ea-33326e1979f0/manager/0.log"
Sep 09 21:16:38.867361 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:38.867328    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8prvh/perf-node-gather-daemonset-sgfwp"
Sep 09 21:16:39.341970 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:39.341891    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dscmd_61c527f5-57d1-404a-bffa-bcbb1433d443/manager/0.log"
Sep 09 21:16:39.420606 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:39.420570    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-845f9dff7c-ft8v9_2359d829-685c-4f24-8b2a-766e95cd1aae/manager/0.log"
Sep 09 21:16:39.620482 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:39.620389    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859cf57674-5fdk2_394ec89c-e336-4d95-a554-5e7080fe9862/manager/0.log"
Sep 09 21:16:39.643150 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:39.643121    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859cf57674-5fdk2_394ec89c-e336-4d95-a554-5e7080fe9862/copy-manifests/0.log"
Sep 09 21:16:39.879362 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:39.879282    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c24thp_b5d19ecc-2831-40b5-a6fa-9f444be087f8/istio-proxy/0.log"
Sep 09 21:16:40.451424 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:40.451397    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65d657c8d5-cww2l_296ba124-bae7-497c-bf5b-1331e85f4b77/kube-auth-proxy/0.log"
Sep 09 21:16:40.493386 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:40.493348    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jgz67_3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c/istio-proxy/0.log"
Sep 09 21:16:40.497002 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:40.496975    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jgz67_3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c/istio-proxy/1.log"
Sep 09 21:16:40.560812 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:40.560788    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8n4mh_85871292-25fc-4b5d-ad75-c0df6e970f08/payload-processing/0.log"
Sep 09 21:16:42.197843 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:42.197809    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c24thp_b5d19ecc-2831-40b5-a6fa-9f444be087f8/istio-proxy/0.log"
Sep 09 21:16:42.781421 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:42.781394    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65d657c8d5-cww2l_296ba124-bae7-497c-bf5b-1331e85f4b77/kube-auth-proxy/0.log"
Sep 09 21:16:42.826941 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:42.826902    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jgz67_3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c/istio-proxy/0.log"
Sep 09 21:16:42.831870 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:42.831812    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jgz67_3e2ed2bd-5842-44e8-a0dc-4c81cf21ce9c/istio-proxy/1.log"
Sep 09 21:16:42.900495 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:42.900458    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8n4mh_85871292-25fc-4b5d-ad75-c0df6e970f08/payload-processing/0.log"
Sep 09 21:16:47.715372 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:47.715340    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-86drj_ad1a3564-db90-40a2-9318-bef1ab32346b/kube-multus/0.log"
Sep 09 21:16:48.051311 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:48.051278    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5pdg_9cf69c29-0898-41f4-8484-99472336cd97/kube-multus-additional-cni-plugins/0.log"
Sep 09 21:16:48.076626 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:48.076595    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5pdg_9cf69c29-0898-41f4-8484-99472336cd97/egress-router-binary-copy/0.log"
Sep 09 21:16:48.100589 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:48.100562    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5pdg_9cf69c29-0898-41f4-8484-99472336cd97/cni-plugins/0.log"
Sep 09 21:16:48.124259 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:48.124209    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5pdg_9cf69c29-0898-41f4-8484-99472336cd97/bond-cni-plugin/0.log"
Sep 09 21:16:48.147457 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:48.147423    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5pdg_9cf69c29-0898-41f4-8484-99472336cd97/routeoverride-cni/0.log"
Sep 09 21:16:48.178083 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:48.178049    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5pdg_9cf69c29-0898-41f4-8484-99472336cd97/whereabouts-cni-bincopy/0.log"
Sep 09 21:16:48.200195 ip-10-0-140-126 kubenswrapper[2550]: I0909 21:16:48.200155    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5pdg_9cf69c29-0898-41f4-8484-99472336cd97/whereabouts-cni/0.log"